Moving picture coding method, moving picture coding apparatus, moving picture decoding method, moving picture decoding apparatus, and moving picture coding and decoding apparatus

ABSTRACT

A moving picture coding method includes: coding a coding target block using a motion vector; generating motion vector predictors; and coding the motion vector using one of the motion vector predictors generated in the generating of the motion vector predictors. In the generating of the motion vector predictors, a replacement vector which replaces a temporal motion vector predictor is added to the motion vector predictors when it is impossible to obtain the temporal motion vector predictor from a block which is included in a coded picture different from the coding target picture and corresponds to the coding target block.

FIELD

One or more exemplary embodiments of the present disclosure relates to amoving picture coding method and a moving picture decoding method.

BACKGROUND

In a video coding process, the amount of information is generallycompressed utilizing redundancy in a spatial direction or a temporaldirection of a video. Here, transform into a frequency domain isgenerally used as a method utilizing redundancy in a spatial direction.On the other hand, inter picture prediction (hereinafter referred to asinter prediction) is used as a method utilizing redundancy in a temporaldirection.

When coding a coding target picture in an inter prediction codingprocess, a coded picture located forward or backward of the codingtarget picture in display time order is used as a reference picture. Amotion vector is derived from the reference picture by performing motionestimation for the coding target picture. Then, the redundancy in thetemporal direction is removed by calculating the difference between theprediction image data obtained by performing motion compensation basedon the motion vector and the image data of the coding target picture.Here, in the motion estimation, the value of difference between a codingtarget block in the coding target picture and each of the blocks in thereference picture is calculated, and the block having the minimumdifference value in the reference picture is determined as a referenceblock. The motion vector is estimated using the coding target block andthe reference block.

In the standardized video coding standard called H.264, three types ofpictures called I-picture, P-picture, and B-picture are used to compressthe amount of information. An I-picture is a picture for which no interprediction coding process is performed, in other words, for which onlyintra picture prediction (hereinafter referred to as intra prediction)coding processes are performed. A P-picture is a picture for which interprediction coding is performed with reference to only a coded picturelocated forward or backward of a coding target picture in display timeorder. A B-picture is a picture for which inter prediction coding isperformed with reference to two coded pictures each located forward orbackward of a coding target picture in display time order.

In addition, the video coding standard called H.264 supports motionvector estimation modes for coding the value of difference betweenprediction image data and a coding target block and a motion vector usedto generate the prediction image data, as coding modes for performinginter prediction on each of coding target blocks in a B-picture. As themotion vector estimation modes, the following directions can beselected: a bidirectional prediction for generating a prediction imagewith reference to two coded pictures located forward or backward of acoding target picture; and a unidirectional prediction for generating aprediction image with reference to a coded picture located forward orbackward of a coding target picture.

In addition, in the video coding standard called H.264, it is possibleto select a coding mode called a temporal motion vector predictor modewhen deriving a motion vector in coding of a B-picture. The interprediction coding method in the temporal motion vector predictor mode isdescribed with reference to FIG. 19. FIG. 19 is an illustration ofmotion vectors in the temporal motion vector predictor mode, and shows acase of coding a block a in a picture B2 using the temporal motionvector predictor mode.

In this case, a motion vector vb which is of a block b co-located withthe block a and in the picture P3 is used. The picture P3 is a referencepicture located backward of the picture B2. The motion vector vb is amotion vector used in the coding of the block b, and shows reference toa picture P1. The block a is coded by bidirectional prediction using areference block obtained from the picture P1 which is a forwardreference picture and a picture P3 which is a backward reference pictureusing a motion vector parallel to the motion vector vb. In other words,the motion vectors used in the coding of the block a is a motion vectorvat in relation to the picture P1 and a motion vector vat in relation tothe picture P3.

CITATION LIST Non Patent Literature

[NPL 1]

-   ITU-T Recommendation H.264 “Advanced Video Coding for generic    audiovisual services”, March, 2010

SUMMARY Technical Problem

However, in the conventional temporal motion vector predictor mode, wheninformation of a reference picture having information such as a motionvector to be used to calculate a temporal motion vector predictor islost, for example, due to a packet loss in streaming distribution, it isimpossible to calculate a correct temporal motion vector predictor,resulting in a deteriorated decoded image. Furthermore, the errorpropagates to pictures which refer to the decoded image, which mayresult in stoppage of the decoding process. For example, wheninformation of the reference picture P3 in FIG. 19 is lost, it isimpossible to calculate a temporal motion vector predictor for thepicture B2. This may makes it impossible to perform correct decoding ofthe picture B2, which may result in stoppage of the decoding process.

One non-limiting embodiment has been made in view of this to provide amoving picture coding method and a moving picture decoding method foreffectively preventing error propagation in decoding processes.

Solution to Problem

In one general aspect, the techniques disclosed here feature a movingpicture coding method for performing inter prediction coding on a codingtarget block included in a coding target picture, the moving picturecoding method including: coding the coding target block using a motionvector; generating a plurality of motion vector predictors; and codingthe motion vector using one of the plurality of motion vector predictorsgenerated in the generating, wherein, in the generating, a replacementvector which replaces a temporal motion vector predictor is included inthe plurality of motion vector predictors when it is impossible toobtain the temporal motion vector predictor from a block which isincluded in a coded picture different from the coding target picture andcorresponds to the coding target block.

These general and specific aspects may be implemented using a system, anapparatus, an integrated circuit, a computer program, or acomputer-readable recording medium such as a CD-ROM, or any combinationof systems, apparatuses, integrated circuits, computer programs, orcomputer-readable recording media.

Additional benefits and advantages of the disclosed embodiments will beapparent from the Specification and Drawings. The benefits and/oradvantages may be individually obtained by the various embodiments andfeatures of the Specification and Drawings, which need not all beprovided in order to obtain one or more of such benefits and/oradvantages.

Advantageous Effects

According to one or more exemplary embodiments of the moving picturecoding method, it is possible to prevent decoding error propagationwhile suppressing decrease in coding efficiency by selectively includinga temporal motion vector predictor or a replacement vector in motionvector predictor candidates.

BRIEF DESCRIPTION OF DRAWINGS

These and other advantages and features will become apparent from thefollowing description thereof taken in conjunction with the accompanyingDrawings, by way of non-limiting examples of embodiments disclosedherein.

FIG. 1 is a block diagram of a moving picture coding apparatus accordingto Embodiment 1.

FIG. 2 is a diagram showing a schematic flow of processes in a movingpicture coding method according to Embodiment 1.

FIG. 3 shows examples of motion vector predictor candidates.

FIG. 4 shows an example of a code table for use in performing variablelength coding on the motion vector predictor indices.

FIG. 5 shows a flow of determining a motion vector predictor candidate.

FIG. 6 is a conceptual diagram showing processes of reading from andwriting to a colPic memory and a global vector storing unit.

FIG. 7A is a diagram showing detailed processing flow of Step S11 inFIG. 2.

FIG. 7B shows examples of B-pictures which are referred to by otherpictures.

FIG. 8 is a diagram showing detailed processing flow of Step S17 in FIG.2.

FIG. 9 is a diagram showing detailed processing flow of Steps S13 andS14 in FIG. 2.

FIG. 10A is a diagram showing an exemplary method of deriving a motionvector predictor candidate using a forward reference motion vector.

FIG. 10B is a diagram showing an exemplary method of deriving a motionvector predictor candidate using a backward reference motion vector.

FIG. 11A is a diagram showing an exemplary method of deriving a motionvector predictor candidate using a backward reference motion vector.

FIG. 11B is a diagram showing an exemplary method of deriving a motionvector predictor candidate using a forward reference motion vector.

FIG. 12 is a block diagram of a moving picture decoding apparatusaccording to Embodiment 2.

FIG. 13 is a diagram showing a schematic flow of processes in a movingpicture decoding method according to Embodiment 2.

FIG. 14 is a diagram showing examples of syntax of a bitstream in themoving picture decoding method according to Embodiment 2.

FIG. 15 is a block diagram of a moving picture coding apparatusaccording to Variation of Embodiment 1.

FIG. 16 is a flowchart of operations in a moving picture coding methodaccording to Variation of Embodiment 1.

FIG. 17 is a diagram showing pictures in a base view and pictures in adependent view.

FIG. 18 is a block diagram of a moving picture decoding apparatusaccording to Variation of Embodiment 2.

FIG. 19 is an illustration of motion vectors in a temporal motion vectorpredictor mode.

FIG. 20 shows an overall configuration of a content providing system forimplementing content distribution services.

FIG. 21 shows an overall configuration of a digital broadcasting system.

FIG. 22 shows a block diagram illustrating an example of a configurationof a television.

FIG. 23 shows a block diagram illustrating an example of a configurationof an information reproducing/recording unit that reads and writesinformation from and on a recording medium that is an optical disk.

FIG. 24 shows an example of a configuration of a recording medium thatis an optical disk.

FIG. 25A shows an example of a cellular phone.

FIG. 25B is a block diagram showing an example of a configuration of acellular phone.

FIG. 26 illustrates a structure of multiplexed data.

FIG. 27 schematically shows how each stream is multiplexed inmultiplexed data.

FIG. 28 shows how a video stream is stored in a stream of PES packets inmore detail.

FIG. 29 shows a structure of TS packets and source packets in themultiplexed data.

FIG. 30 shows a data structure of a PMT.

FIG. 31 shows an internal structure of multiplexed data information.

FIG. 32 shows an internal structure of stream attribute information.

FIG. 33 shows steps for identifying video data.

FIG. 34 shows an example of a configuration of an integrated circuit forimplementing the moving picture coding method and the moving picturedecoding method according to each of embodiments.

FIG. 35 shows a configuration for switching between driving frequencies.

FIG. 36 shows steps for identifying video data and switching betweendriving frequencies.

FIG. 37 shows an example of a look-up table in which video datastandards are associated with driving frequencies.

FIG. 38A is a diagram showing an example of a configuration for sharinga module of a signal processing unit.

FIG. 38B is a diagram showing another example of a configuration forsharing a module of the signal processing unit.

DESCRIPTION OF EMBODIMENTS

(Outline of One or More Exemplary Embodiments of the Present Disclosure)

A moving picture coding method according to one exemplary embodiment ofthe present disclosure is a method for performing inter predictioncoding on a coding target block included in a coding target picture.More specifically, the moving picture coding method including: codingthe coding target block using a motion vector; generating a plurality ofmotion vector predictors; and coding the motion vector using one of theplurality of motion vector predictors generated in the generating,wherein, in the generating, a replacement vector which replaces atemporal motion vector predictor is included in the plurality of motionvector predictors when it is impossible to obtain the temporal motionvector predictor from a block which is included in a coded picturedifferent from the coding target picture and corresponds to the codingtarget block.

With this structure, it is possible to prevent decoding errorpropagation while suppressing decrease in coding efficiency byselectively including a temporal motion vector predictor or areplacement vector in motion vector predictor candidates.

In addition, for example, in the generating, a motion vector having amotion quantity of 0 may be included, as the replacement vector, in theplurality of motion vector predictors when obtainment of the temporalmotion vector predictor from the coded picture is prohibited.

In addition, for example, in the generating, the number of picturescoded according to the moving picture coding method may be counted, andobtainment of the temporal motion vector predictor from the codedpicture may be prohibited when the coding target picture is coded, thecoding target picture being a picture coded at time at which the numberof coded pictures exceeds a predetermined value.

In addition, for example, the moving picture coding method may be amethod of coding pictures each of which is in a base view or a dependentview included in a multi-view video, and may further include generatinga parallax vector corresponding to parallax between the base view andthe dependent view. In addition, in the generating, the parallax vectormay be included, as the replacement vector, in the plurality of motionvector predictors when the coding target picture is in the dependentview and is a starting picture in a Group Of Pictures (GOP).

In addition, for example, the moving picture coding method may be amethod of coding pictures each of which is in a base view or a dependentview included in a multi-view video, and further includes generating aparallax vector corresponding to parallax between the base view and thedependent view. In addition, for example, in the generating, theparallax vector may be included, as the replacement vector, in theplurality of motion vector predictors when obtainment of the temporalmotion vector predictor from the coded picture is prohibited.

In addition, for example, the parallax vector may be calculated using amotion vector obtained when inter-view prediction is performed on eachof blocks included in the picture in the dependent view, using thepicture included in the base view and corresponding to the picture inthe dependent view. In addition, in the generating, the parallax vectormay be included, as the replacement vector, in the plurality of motionvector predictors, the parallax vector being used when coding a startingpicture in a GOP immediately before the GOP including the coding targetpicture.

In addition, for example, the parallax vector may be calculated using amotion vector obtained when inter-view prediction is performed on eachof blocks included in the dependent view, using the picture included inthe base view and corresponding to the picture in the dependent view. Inaddition, in the generating, the parallax vector may be included, as thereplacement vector, in the plurality of motion vector predictors, theparallax vector being used when coding a picture coded immediatelybefore the coded picture.

The moving picture decoding method according to one exemplary embodimentof the present disclosure is a method for performing inter predictiondecoding on a decoding target block in a decoding target picture. Morespecifically, the moving picture decoding method includes: generating aplurality of motion vectors; decoding the motion vector using one of theplurality of motion vector predictors generated in the generating; anddecoding the decoding target block using the motion vector decoded inthe decoding. In the generating, a replacement vector which replaces atemporal motion vector predictor is included in the plurality of motionvector predictors when it is impossible to obtain the temporal motionvector predictor from a block which is included in a decoded picturedifferent from the decoding target picture and corresponds to thedecoding target block.

The moving picture coding method according to one exemplary embodimentof the present disclosure is a method of coding the coding target blockusing a reference motion vector of a reference block included in areference picture different from a coding target picture including thecoding target block. The reference block in the reference picture isco-located with the coding target block in the coding target picture.The image coding method includes: determining a value of a predeterminedflag indicating whether to use a first reference motion vector of thereference block or a second reference motion vector of the referenceblock at the time of motion vector coding on the coding target block;assigning a bitstream a third reference motion vector calculated fromthe second reference motion vector when the predetermined flag indicatesuse of the second reference motion vector of the reference picture;coding a motion vector for the coding target block according to thevalue of the predetermined flag; and a flag assigning unit of assigningthe bitstream the predetermined flag.

In addition, for example, the determining may include: counting thenumber of coded pictures among the coding target pictures; and, when thenumber of coded pictures is smaller than a predetermined value,determining use of the first reference motion vector of the referenceblock at the time of motion vector coding on the coding target block;and when the number of coded pictures among the coding target picturesis larger than or equal to the predetermined value, determining use ofthe second reference motion vector of the reference block at the time ofmotion vector coding on the coding target block, and resetting thenumber.

In addition, for example, the second reference motion vector of thereference picture may be calculated from an average value of motionvectors of coded blocks in the reference picture.

In addition, for example, the second reference motion vector of thereference picture may be calculated from a motion vector which appearsmost frequently from among motion vectors of coded blocks in thereference picture.

In addition, for example, the coding may include, when the referenceblock includes two or more reference motion vectors: selecting one ofthe reference motion vectors, based on whether the reference picture islocated forward or backward of the coding target picture; and coding themotion vector for the coding target block using the determined referencemotion vector.

In addition, for example, when the reference block includes forward andbackward reference motion vectors, in the selecting, the forwardreference motion vector among the forward and backward reference motionvectors may be selected when the coding target block is located forwardof the reference block; and the backward reference motion vector amongthe forward and backward reference motion vectors may be selected whenthe coding target block is located backward of the reference block.

In addition, for example, when the reference block includes one of theforward and backward reference motion vectors, in the selecting, one ofthe forward and backward reference motion vectors which is included inthe reference block may be selected irrespective of a positionalrelationship between the reference block and the coding target block.

The moving picture decoding method according to one exemplary embodimentof the present disclosure is a method for decoding the decoding targetblock using a reference motion vector of a reference block included in areference picture different from a decoding target picture including thedecoding target block. The reference block in the reference picture isco-located with the decoding target block in the decoding targetpicture. The image decoding method includes: decoding a value of apredetermined flag indicating whether to use a first reference motionvector of the reference block or to use a second reference motion vectorof the reference picture at the time of motion vector decoding for thedecoding target block; decoding, from a bitstream, a third referencemotion vector calculated from the second reference motion vector whenthe predetermined flag indicates use of the second reference motionvector of the reference picture; and decoding a motion vector of thedecoding target block according to the value of the predetermined flag.

In addition, for example, the decoding may include, when the referenceblock includes two or more reference motion vectors: selecting one ofthe reference motion vectors, based on whether the reference picture islocated forward or backward of the decoding target picture; and decodingthe motion vector for the decoding target block using the determinedreference motion vector.

In addition, for example, when the reference block includes forward andbackward reference motion vectors, in the selecting, the forwardreference motion vector among the forward and backward reference motionvectors may be selected when the decoding target block is locatedforward of the reference block; and the backward reference motion vectoramong the forward and backward reference motion vectors may be selectedwhen the decoding target block is located backward of the referenceblock.

In addition, for example, when the reference block includes one of theforward and backward reference motion vectors, in the selecting, one ofthe forward and backward reference motion vectors which is included inthe reference block may be selected irrespective of a positionalrelationship between the reference block and the decoding target block.

In addition, for example, the second reference motion vector of thereference picture may be calculated from an average value of motionvectors of decoded blocks in the reference picture.

In addition, for example, the second reference motion vector of thereference picture may be calculated from a motion vector which appearsmost frequently from among motion vectors of decoded blocks in thereference picture.

These general and specific aspects may be implemented using a system, anapparatus, an integrated circuit, a computer program, or acomputer-readable recording medium such as a CD-ROM, or any combinationof systems, apparatuses, integrated circuits, computer programs, orcomputer-readable recording media.

Hereinafter, exemplary embodiments are described in detail withreference to the drawings.

It is to be noted that each of the exemplary embodiments described belowshows a general or specific example. The numerical values, shapes,materials, structural elements, the arrangement and connection of thestructural elements, steps, the processing order of the steps etc. shownin the following exemplary embodiments are mere examples, and thereforedo not limit the scope of the appended Claims and their equivalents. Inaddition, among the structural elements in the following exemplaryembodiments, structural elements not recited in any one of theindependent claims are described as arbitrary structural elements.

It is to be noted that the term “coding” may be used to mean the term“encoding”.

Embodiment 1

FIG. 1 is a block diagram of a moving picture coding apparatus whichperforms a moving picture coding method according to Embodiment 1.

As shown in FIG. 1, the moving picture coding apparatus 100 includes: asubtractor 101, an orthogonal transform unit 102, a quantization unit103, an inverse quantization unit 104, an inverse orthogonal transformunit 105, an adder 106, a block memory 107, a frame memory 108, an intraprediction unit 109, an inter prediction unit 110, a switch 111, aninter prediction control unit 112, a picture type determining unit 113,a temporal motion vector predictor calculating unit 114, a colPic memory115, a global vector storing unit 116, a co-located block informationdetermining unit 117, and a variable length encoder 118.

The subtractor 101 obtains an input image stream including a codingtarget block from outside of the apparatus, obtains a prediction blockfrom the switch 111, and outputs a residual block obtained bysubtracting the prediction block from a coding target block to theorthogonal transform unit 102.

The orthogonal transform unit 102 transforms the residual block obtainedfrom the subtractor 101 from an image domain to a frequency domain, andoutputs the transform coefficients to the quantization unit 103. Thequantization unit 103 quantizes the transform coefficients obtained fromthe quantization unit 103, and outputs the quantized coefficients to theinverse quantization unit 104 and the variable length encoder 118.

The inverse quantization unit 104 performs inverse quantization on thequantized coefficients obtained from the quantization unit, and outputsthe reconstructed transform coefficients to the inverse orthogonaltransform unit 105. The inverse orthogonal transform unit 105 transformsthe reconstructed transform coefficients obtained from the inversequantization unit 104 from the frequency domain to the image domain, andoutputs the reconstructed residual block to the adder 106.

The adder 106 adds the reconstructed residual block obtained from theinverse orthogonal transform unit 105 and the prediction block obtainedfrom the switch 111, and outputs the reconstructed target block to theblock memory 107 and the frame memory 108. The block memory 107 storesthe reconstructed input image stream on a block-by-block basis. Theframe memory 108 stores the reconstructed input image stream on aframe-by-frame basis.

The picture type determining unit 113 determines the picture type to beused for the coding of the input image stream from among I-picture,B-picture, and P-picture, and generates picture type information. Thepicture type determining unit 113 outputs the generated picture typeinformation to the switch 111, the inter prediction control unit 112,the co-located block information determining unit 117, and the variablelength encoder 118.

The intra prediction unit 109 performs intra prediction on the codingtarget block using the reconstructed block-based input image streamstored in the block memory 107 to generate a prediction block, andoutputs the generated prediction block to the switch 111. The interprediction unit 110 performs inter prediction on the coding target blockusing the reconstructed frame-based input image stream stored in theframe memory 108 and a motion vector derived through motion estimationto generate a prediction block, and outputs the generated predictionblock to the switch 111.

The switch 111 outputs the prediction block generated by the intraprediction unit 109 or the prediction block generated by the interprediction unit 110 to the subtractor 101 and the adder 106. Forexample, the switch 111 outputs one of the two prediction blocks whichhas a smaller coding cost.

The co-located block information determining unit 117 determines whetheror not to prohibit use of a co-located block. The co-located blockinformation determining unit 117 generates, for each of pictures, aco-located block use prohibition flag indicating the result of thedetermination, and outputs the flag to the temporal motion vectorpredictor calculating unit 114 and the variable length encoder 118. Thisco-located block use prohibition flag is included in the bitstream(typically, in a picture header or a slice header).

In addition, the co-located block information determining unit 117determines, as a co-located block, one of a block (hereinafter referredto as a forward reference block) which is included in a picture locatedforward of the coding target picture in display time order and a block(hereinafter referred to as a backward reference block) which isincluded in a picture located backward of the coding target picture indisplay time order. In other words, the forward reference block is ablock included in a reference picture which is determined by a referencepicture list L0. In addition, the backward reference block is a blockincluded in a reference picture which is determined by a referencepicture list L1.

The co-located block information determining unit 117 generates, foreach of the pictures, a co-located reference block direction flagindicating the result of the determination, and outputs the flag to thetemporal motion vector predictor calculating unit 114 and the variablelength encoder 118. This co-located reference block direction flag isincluded in the bitstream (typically, in a picture header or a sliceheader). In addition, when a value indicating “prohibition” is set tothe co-located block use prohibition flag, the co-located referenceblock direction flag may be omitted.

Here, a co-located block is a block which is in a picture different froma picture including a coding target block and is co-located with thecoding target block. It is to be noted that the coding target block andthe co-located block are not always need to be precisely co-located witheach other in the pictures. For example, a block surrounding(neighboring) the block which is in a picture different from the codingtarget picture and is co-located with the coding target block may bedetermined as a co-located block.

The temporal motion vector predictor calculating unit 114 derives amotion vector predictor candidate using colPic information such as amotion vector of a co-located block stored in the colPic memory 115 anda global motion vector of a colPic picture stored in the global vectorstoring unit, according to the value of the co-located block useprohibition flag obtained from the co-located block informationdetermining unit 117.

More specifically, when the co-located block use prohibition flag is on(prohibition), the temporal motion vector predictor calculating unit 114adds the global motion vector (replacement vector) read from the globalvector storing unit 116 to the motion vector predictor candidates. Onthe other hand, when the co-located block use prohibition flag is off(allowance), the temporal motion vector predictor calculating unit 114adds the temporal motion vector predictor calculated using the colPicinformation read out from the colPic memory 115 to the motion vectorpredictor candidates.

In addition, the temporal motion vector predictor calculating unit 114assigns the motion vector predictor added as a candidate a motion vectorpredictor index value. The temporal motion vector predictor calculatingunit 114 outputs the motion vector predictor added as the candidate andthe motion vector predictor index to the inter prediction control unit112. On the other hand, when the co-located block does not have anymotion vector, the temporal motion vector predictor calculating unit 114stops the motion vector derivation in a temporal motion vector predictormode or derives a motion vector having a motion quantity of 0 as amotion vector predictor candidate. In addition, the temporal motionvector predictor calculating unit 114 outputs the global motion vectorto the variable length encoder 118.

The inter prediction control unit 112 determines that a motion vector iscoded using a motion vector predictor candidate having the minimumdifference from a motion vector derived through motion estimation fromamong a plurality of motion vector predictor candidates. Here, forexample, a difference shows the value of difference between the motionvector predictor candidate and the motion vector derived through themotion estimation.

In addition, the inter prediction control unit 112 generates a motionvector predictor index corresponding to the determined motion vectorpredictor on a block-by-block basis. Next, the inter prediction controlunit 112 transmits the motion vector predictor index, and the value ofdifference between the motion vector and the motion vector predictor tothe variable length encoder. In addition, the inter prediction controlunit 112 transmits colPic information including the motion vector etc.for the coding target block to the colPic memory 115. In addition, theinter prediction control unit 112 transmits the motion vector etc. forthe coding target block to the global vector storing unit 116.

The colPic information including the motion vector etc. for the codingtarget block is stored in the colPic memory 115 for use as a vectorpredictor at the time of coding a next picture. A global motion vectorcalculated from the motion vectors for the coding target blocks in thewhole picture is stored in the global vector storing unit 116 for use asa vector predictor at the time of coding the next picture.

The variable length encoder 118 generates a bitstream by performing avariable length coding process on: the quantized coefficients obtainedfrom the quantization unit 103; the motion vector predictor index, thevalue of difference between the motion vector and the motion vectorpredictor obtained from the inter prediction control unit 112; thepicture type information obtained from the picture type determining unit113; the co-located block use prohibition flag and the co-locatedreference block direction flag obtained from the co-located blockinformation determining unit 117; and the temporal global motion vectorpredictor obtained from the temporal motion vector predictor calculatingunit 114.

FIG. 2 is a diagram showing a schematic flow of processes in a movingpicture coding method according to Embodiment 1.

The co-located block information determining unit 117 determinesco-located block information such as co-located block use prohibitionflag and co-located reference block direction flag according to alater-described method when deriving a motion vector predictor candidatein the temporal motion vector predictor mode (S11).

Next, the temporal motion vector predictor calculating unit 114determines whether or not the co-located block use prohibition flag ison (prohibition) (S12). When the result of the determination is true(Yes in S12), the temporal motion vector predictor calculating unit 114reads a global motion vector from the global vector storing unit 116 andadds the read-out global motion vector to header information such as apicture header (S13).

Next, the temporal motion vector predictor calculating unit 114 adds theglobal motion vector, as a replacement vector which replaces thetemporal motion vector predictor, to motion vector predictor candidates.In addition, the temporal motion vector predictor calculating unit 114assigns the motion vector predictor added as the candidate a motionvector predictor index value.

On the other hand, when the co-located block use prohibition flag is off(No in S12), the temporal motion vector predictor calculating unit 114reads colPic information including a reference motion vector etc. for aco-located block from the colPic memory according to the co-locatedblock information, and adds a temporal motion vector predictorcalculated using the reference motion vector of the co-located block tothe motion vector predictor candidates (S17). In addition, the temporalmotion vector predictor calculating unit 114 assigns the motion vectorpredictor added as the candidate a motion vector predictor index value.

In general, a smaller motion vector predictor index value shows asmaller amount of necessary information. On the other hand, a largermotion vector predictor index value shows a larger amount of necessaryinformation. Accordingly, coding efficiency is increased when a smallmotion vector predictor index value is assigned to a motion vectorhaving a high possibility of becoming a highly accurate motion vector.

Next, the inter prediction unit 110 performs inter prediction using amotion vector derived through motion estimation to generate a predictionblock for the coding target block. Subsequently, the subtractor 101, theorthogonal transform unit 102, the quantization unit 103, and thevariable length encoder 118 operate to perform coding processes on thecoding target block using the prediction block generated by the interprediction unit 110.

In addition, the inter prediction control unit 112 codes a motion vectorusing a motion vector predictor having the minimum difference from themotion vector selected from among the plurality of motion vectorpredictor candidates. The inter prediction control unit 112 determines,to be differences, the values of differences between the respectivemotion vector predictor candidates and the motion vector derived throughthe motion estimation, and determines the motion vector predictor havingthe minimum difference to be a motion vector predictor for use in thecoding of the motion vector.

Next, the inter prediction control unit 112 outputs the motion vectorpredictor index corresponding to the selected motion vector predictor,and the difference information between the motion vector and the motionvector predictor to the variable length encoder 118. The variable lengthencoder 118 performs variable length coding on the motion vector indexand the difference information obtained from the inter predictioncontrol unit 112, and includes them in the bitstream.

Next, the inter prediction control unit 112 stores the colPicinformation including the motion vector etc. used in the interprediction in the colPic memory 115. In order to calculate a temporalmotion vector predictor for the coding target block, a motion vector ina reference picture, an index value of the reference picture, and aprediction direction etc. are stored in the colPic memory 115. Inaddition, the inter prediction control unit 112 stores a motion vectoretc. used in the inter prediction in the global vector storing unit 116(S16).

FIG. 3 shows examples of motion vector predictor candidates. A motionvector A (MV_A) is a motion vector for a neighboring block A located tothe left of the coding target block. A motion vector B (MV_B) is amotion vector for a neighboring block B located above the coding targetblock. A motion vector C (MV_C) is a motion vector of a neighboringblock C located right above the coding target block. In addition, Median(of MV_A, MV_B, MV_C) shows a median value of the motion vectors A, B,and C. Here, the median value is derived using, for example, Expressions1 to 3 as shown below.

[Math. 1]Median(x,y,z)=x+y+z−Min(x,Min(y,z))−Max(x,Max(y,z))   (Expression 1)

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 2} \right\rbrack & \; \\{{{Min}\left( {x,y} \right)} = \left\{ \begin{matrix}x & \left( {x \leq y} \right) \\y & \left( {x > y} \right)\end{matrix} \right.} & \left( {{Expression}\mspace{14mu} 2} \right)\end{matrix}$

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 3} \right\rbrack & \; \\{{{Max}\left( {x,y} \right)} = \left\{ \begin{matrix}x & \left( {x \geq y} \right) \\y & \left( {x < y} \right)\end{matrix} \right.} & \left( {{Expression}\mspace{14mu} 3} \right)\end{matrix}$

The motion vector predictor index values are as follows: the valuecorresponding to the Median (of MV_A, MV_B, MV_C) is 0; the valuecorresponding to the motion vector A is 1; the value corresponding tothe motion vector B is 2; the value corresponding to the motion vector Cis 3; and the value corresponding to the temporal motion vectorpredictor (or a replacement vector) is 4. How to assign values to themotion vector predictor indices are not limited to this example.

FIG. 4 shows an example of a code table for use in performing variablelength coding on the motion vector predictor indices. In the example ofFIG. 4, codes are assigned to the motion vector predictor indices suchthat a code having a shorter code length is assigned to a motion vectorpredictor index having a smaller value. Accordingly, it is possible toincrease the coding efficiency by assigning a small motion vector indexvalue to a motion vector predictor candidate having a high possibilityof yielding a high prediction accuracy.

FIG. 5 is a diagram of a motion vector predictor candidate determinationflow performed by the inter prediction control unit 112. According tothe flow shown in FIG. 5, the motion vector predictor candidate havingthe minimum difference from the motion vector derived through the motionestimation is determined to be the motion vector predictor which is usedwhen the motion vector is coded. Next, the information about thedifference between the motion vector and the motion vector predictor andthe motion vector predictor index indicating the determined motionvector predictor are variable-length coded and included in thebitstream.

More specifically, first, the inter prediction control unit 112initializes the motion vector predictor candidate index mvp_idx and theminimum motion vector difference (S21). Next, the inter predictioncontrol unit 112 compares a motion vector predictor candidate indexmvp_idx and the number of motion vector predictors (the number ofrecords in a table shown in FIG. 3) (S22).

When mvp_idx<the number of motion vector predictors is satisfied (Yes inS22), the inter prediction control unit 112 calculates a motion vectordifference (difference information) using one of the plurality of motionvector predictor candidates (S23). For example, the inter predictioncontrol unit 112 calculates a motion vector difference by subtracting amotion vector predictor having a motion vector predictor index of 0 inFIG. 3 from a motion vector used to code a coding target block.

Next, the inter prediction control unit 112 compares the motion vectordifference calculated in Step S23 and the minimum motion vectordifference (S24). When a motion vector difference<the minimum motionvector difference is satisfied (Yes in S24), the inter predictioncontrol unit 112 sets (overwrites) the motion vector differencecalculated in Step S23 to the minimum motion vector difference, and sets(overwrites) the current mvp_idx to the motion vector predictor index(S25). On the other hand, when a motion vector difference the minimummotion vector difference is satisfied (No in S24), Step S25 is skipped.

The inter prediction control unit 112 increments the mvp_idx by 1 (S26),and repeatedly executes the above-described processes by the number oftimes corresponding to the number of motion vector predictors (Steps S22to S26). The inter prediction control unit 112 outputs the minimummotion vector difference and a value which is set to the motion vectorpredictor index to the variable length encoder 118 at the time whenmvp_idx=the number of motion vector predictor candidates is satisfied(S22), to complete the processes in FIG. 5 (S27).

FIG. 6 is a conceptual diagram showing processes of reading from andwriting into the colPic memory 115 and the global vector storing unit116 shown in FIG. 1. In FIG. 6, a motion vector mvCol1 in a predictiondirection 1 and a motion vector mvCol2 in a prediction direction 2 ofthe co-located block in a co-located picture colPic are stored in thecolPic memory 115 and the global vector storing unit 116.

Here, the co-located block is a block which is in the co-located picturecolPic and is co-located with the coding target block. In addition,whether the co-located picture colPic is the one located backward of thecoding target picture or the one located forward of the coding targetpicture is switched according to a co-located reference block directionflag. When the coding target block is coded, the colPic informationincluding the motion vector etc, stored in the colPic memory 115 or theglobal motion vector in the global vector storing unit 116 is read outaccording to the co-located block use prohibition flag, and is added tothe motion vector predictor candidates.

The motion vector predictor candidate is used to code a motion vectorfor the coding target block. Embodiment 1 is described taking theexample where the prediction direction 1 and the prediction direction 2are determined to be the forward reference direction and the backwardreference direction, respectively. However, it is to be noted that theprediction direction 1 and the prediction direction 2 may be determinedto be the backward reference direction and the forward referencedirection, respectively, or both of the prediction directions 1 and 2may be determined to be either the forward reference direction or thebackward reference direction.

The global vector storing unit 116 stores the global motion vectorcalculated from the motion vectors for the coding target blocks of thecoding target picture. For example, it is conceivable to determine, tobe the global motion vector, an average value of the motion vectors ineach prediction direction at the time of inter prediction coding of thewhole coding target picture. Embodiment 1 has been described taking thenon-limiting example of using, as the global vector, the average valueof motion vectors for the coding target blocks of the coding targetpicture.

However, for example, it is also possible to determine, as the globalmotion vector, a median value or a weighted average value of motionvectors at the time of performing inter prediction coding on the codingtarget blocks of the coding target picture. Alternatively, it is alsopossible to determine, as the global motion vector, the value of amotion vector having a highest appearance frequency among motion vectorsat the time of performing inter prediction coding on the coding targetblocks of the coding target picture. Alternatively, it is also possibleto determine, as the global motion vector, the value of a motion vectorwhich refers to a closet picture in display order among motion vectorsat the time of the inter prediction coding of the coding target blocksof the coding target picture.

FIG. 7A is a diagram showing detailed processing flow of Step S11 inFIG. 2. Hereinafter, FIG. 7A is described.

First, the co-located block information determining unit 117 determineswhether or not to use the temporal motion vector predictor mode using aco-located block for the coding target picture (S31). Next, theco-located block information determining unit 117 generates, on apicture-by-picture basis, a co-located block use prohibition flagindicating whether or not use of a co-located block is allowed (atemporal direct mode is allowed), and outputs the co-located block useprohibition flag to the variable length encoder 118.

For example, in streaming distribution or the like, it is conceivablethat a co-located block use prohibition flag is on at certain intervalsin order to reduce propagation of a decoding error due to the temporalmotion vector predictor mode. As an example for achieving this, a methodis possible which involves preparing a counter for counting the numberof coded pictures among the coding target pictures, turning off aco-located block use prohibition flag when the number of coded picturesis smaller than a threshold value, and when the number of coded picturesamounts to the threshold value, turning on co-located block useprohibition flag to reset the counter to 0.

In addition, for example, a method is possible which is intended toreduce decoding error propagation by turning on a co-located block useprohibition flag for each of pictures which can be reference targets(such as a P-picture, and a B-picture which can be referred to byanother picture) and turning off a co-located block use prohibition flagfor a picture which cannot be a reference target (such as a B-picturewhich cannot be referred to by any other picture). In this way, it ispossible to effectively reduce such decoding error propagation byturning on the co-located block use prohibition flag for each picturewhich is referred to by another picture.

Next, the co-located block information determining unit 117 determines,to be a co-located block, one of a forward reference block and abackward reference block (S32). As a conceivable example, the co-locatedblock information determining unit 117 determines, to be the co-locatedblock, one of a co-located block (a forward reference block) included ina forward reference picture and a co-located block (a backward referenceblock) included in a backward reference picture which is closer to thecoding target picture in distance in display order. Next, the co-locatedblock information determining unit 117 generates, on apicture-by-picture basis, a co-located reference block direction flagindicating whether the co-located block is the forward reference blockor the backward reference block, and outputs the co-located referenceblock direction flag to the variable length encoder 118.

FIG. 7B shows examples of B-pictures which are referred to by otherpictures. FIG. 7B defines a reference structure composed of a pluralityof layers. A stream is started with an I-picture, and the pictures otherthan the starting I-picture are B-pictures. In addition, in thestructure, the picture belonging to a higher level layer among theplurality of layers refers to a picture belonging to the same levellayer or a picture belonging to a lower level layer.

For example, in FIG. 7B, a picture B1 belonging to a layer 3 refers to apicture I0 belonging to a layer 0 and a picture Br2 belonging to a layer2. In addition, a picture Bf8 belonging to the lowermost level layer 0refers to the picture I0 in the same layer. Here, in the structure, eachof the pictures belonging to the lowermost level layer 0 refers to onlya picture located forward thereof in display order. In this referencestructure, a conceivable method involves turning on a co-located blockuse prohibition flag for each of the pictures belonging to the layer 0which has a high possibility of being referred to by another picture.

FIG. 8 is a diagram showing detailed processing flow of Step S17 in FIG.2. Hereinafter, FIG. 8 is described.

First, the temporal motion vector predictor calculating unit 114 reads,from the colPic memory 115, colPic information including a referencemotion vector in a prediction direction 1 and a reference motion vectorin a prediction direction 2 etc. (S41). Next, the temporal motion vectorpredictor calculating unit 114 determines whether or not the co-locatedblock included in the colPic information has two or more motion vectors(S42). In other words, the temporal motion vector predictor calculatingunit 114 determines whether or not the co-located block has a forwardreference motion vector (mvL0) and a backward reference motion vector(mvL1).

When determining that the co-located block has two or more motionvectors (Yes in S42), the temporal motion vector predictor calculatingunit 114 determines whether or not the co-located block is the backwardreference block (S43). In other words, the temporal motion vectorpredictor calculating unit 114 determines whether or not the pictureincluding the co-located block is located backward of the coding targetpicture in display order.

Next, when determining that the co-located block is the backwardreference block (Yes in S43), the temporal motion vector predictorcalculating unit 114 derives a temporal motion vector predictor using aforward reference motion vector (a motion vector mvL0 corresponding to areference picture in a reference picture list L0) of the co-locatedblock in the temporal motion vector predictor mode (S44). Next, thetemporal motion vector predictor calculating unit 114 adds the temporalmotion vector predictor calculated in Step S44 to the motion vectorpredictor candidates (S45).

On the other hand, when determining that the co-located block is theforward reference block (No in S43), the temporal motion vectorpredictor calculating unit 114 derives a temporal motion vectorpredictor using a backward reference motion vector (a motion vector mvL1corresponding to a reference picture in a reference picture list L1) ofthe co-located block in the temporal motion vector predictor mode (S46),and adds the temporal motion vector predictor to the motion vectorpredictor candidates (S45).

On the other hand, when determining that the co-located block has onlyone of the forward reference block and the backward reference block (Noin S42), the temporal motion vector predictor calculating unit 114determines whether or not the co-located block has a forward referencemotion vector (S47). When determining that the co-located block has theforward reference motion vector (Yes in S47), the temporal motion vectorpredictor calculating unit 114 derives a temporal motion vectorpredictor for the coding target block using the forward reference motionvector of the co-located block (S48), and adds the temporal motionvector predictor to the motion vector predictor candidates (S45).

On the other hand, when determining that the co-located block does nothave any forward reference block (No in S47), the temporal motion vectorpredictor calculating unit 114 determines whether or not the co-locatedblock has a backward reference motion vector (S49). When determiningthat the co-located block has a backward reference block (Yes in S49),the temporal motion vector predictor calculating unit 114 derives atemporal motion vector predictor for the coding target block using thebackward reference motion vector (S50), and adds the temporal motionvector predictor to the motion vector predictor candidates (S45).

On the other hand, when determining that the co-located block does nothave any backward reference motion vector (No in S49), the temporalmotion vector predictor calculating unit 114 terminates the processes inFIG. 8 without adding the temporal motion vector predictor to the motionvector predictor candidates (S51). Alternatively, the temporal motionvector predictor calculating unit 114 may determine the value of atemporal motion vector predictor for the co-located block to be 0(determine the temporal motion vector predictor to be a motion vectorhaving a motion quantity of 0), and add the temporal motion vectorpredictor to the motion vector predictor candidates.

In the processing flow in FIG. 8, whether or not the co-located blockhas a forward reference motion vector is determined in Step S47, andwhether or not the co-located block has a backward reference motionvector is determined in Step S49. However, it is to be noted that theprocessing flow is a non-limiting example. For example, it is also goodto determine whether or not a co-located block has a backward referencemotion vector, and then determine whether or not the co-located blockhas a forward reference motion vector.

FIG. 9 is a diagram showing detailed processing flow of Steps S13 andS14 in FIG. 2. Hereinafter, FIG. 9 is described.

First, the temporal motion vector predictor calculating unit 114 reads,from the global vector storing unit 116, global motion vectorinformation including a global motion vector in the prediction direction1 and/or a global motion vector in the prediction direction 2 (S61).Next, the temporal motion vector predictor calculating unit 114determines whether or not the global motion vector information includestwo or more motion vectors (S62). In other words, the temporal motionvector predictor calculating unit 114 determines whether or not aforward reference motion vector (mvL0) and a backward reference motionvector (mvL1) are included in the global motion vector information.

When determining that the global motion vector information includes twoor more motion vectors (Yes in S62), the temporal motion vectorpredictor calculating unit 114 determines the co-located reference blockdirection is a direction in which a backward reference block is present(S63). When determining that the co-located reference block direction isa direction in which the backward reference block is present (Yes inS63), the temporal motion vector predictor calculating unit 114 selectsa forward reference motion vector in the global motion vectorinformation (S64).

Next, the temporal motion vector predictor calculating unit 114 adds theselected global motion vector to header information such as a pictureheader (outputs it to the variable length encoder 114), and adds theselected global motion vector to the motion vector predictor candidatesfor the coding target block (S65). It is to be noted that the temporalmotion vector predictor calculating unit 114 adds, to the headerinformation, the information for identifying the reference picture whichis referred to by the selected global motion vector (more specifically,the reference picture is referred to by a plurality of motion vectorsfor use in the calculation of the global motion vector). Thisinformation is used in a scaling process which is described later withreference to FIG. 10A to FIG. 11B.

On the other hand, when determining that the co-located reference blockdirection is a direction in which a forward reference block is present(No in S63), the temporal motion vector predictor calculating unit 114selects a backward reference motion vector in the global motion vectorinformation (S66). Next, the temporal motion vector predictorcalculating unit 114 adds the selected global motion vector to headerinformation such as a picture header, and adds the global motion vectorto the motion vector predictor candidates for the coding target block(S65).

In addition, when determining that the global motion vector informationincludes only one of the forward reference block and the backwardreference block (No in S62), the temporal motion vector predictorcalculating unit 114 determines whether or not the global motion vectorinformation includes a forward reference motion vector (S67).

When determining that the global motion vector information includes theforward reference motion vector (Yes in S67), the temporal motion vectorpredictor calculating unit 114 selects the forward reference motionvector for the global motion vector information (S68). Next, thetemporal motion vector predictor calculating unit 114 adds the selectedglobal motion vector to header information such as a picture header, andadds the global motion vector to the motion vector predictor candidatesfor the coding target block (S65).

On the other hand, when determining that the global motion vectorinformation does not include any forward reference block (No in S67),the temporal motion vector predictor calculating unit 114 determineswhether or not the global motion vector information includes a backwardreference motion vector (S69). When determining that the global motionvector information includes the backward reference motion vector (Yes inS69), the temporal motion vector predictor calculating unit 114 selectsthe backward reference motion vector for the global motion vectorinformation (S70). Next, the temporal motion vector predictorcalculating unit 114 adds the selected global motion vector to headerinformation such as a picture header, and adds the global motion vectorto the motion vector predictor candidates for the coding target block(S65).

On the other hand, when determining that the global motion vectorinformation does not include any backward reference motion vector (No inS67), the temporal motion vector predictor calculating unit 114 does notadd the temporal motion vector predictor to motion vector predictorcandidates or determine the value of the global motion vector to be 0(S71). Next, the temporal motion vector predictor calculating unit 114adds the set global motion vector to header information such as apicture header, and adds the global motion vector to the motion vectorpredictor candidates for the coding target block (S65).

In the processing flow in FIG. 9, whether or not the global motionvector information includes a forward reference motion vector isdetermined in Step S67, and whether or not the global motion vectorinformation includes a backward reference motion vector is determined inStep S69. However, it is to be noted that the processing flow is anon-limiting example. For example, it is also good to determine whetheror not the global motion vector information includes the backwardreference motion vector, and then determine whether or not the globalmotion vector information includes the forward reference motion vector.

In addition, as an example, which one of the global motion vectors mvL0and mvL1 is selected is determined according to the co-located referenceblock direction flag in Steps S63 to S66 in FIG. 9. However, one or morenon-limiting embodiments of the present disclosure are not limited tothe example. For example, it is also good to select the global motionvector mvL0 as a motion vector predictor candidate in the referencepicture list L0 and select the global motion vector mvL1 as a motionvector predictor candidate in the reference picture list L1. Thiseliminates the need that a co-located reference block direction flag isadded to a header when the global motion vector is used, which furtherincreases the coding efficiency

Next, a detailed description is given of a scaling method performed whenadding a temporal motion vector predictor to motion vector predictorcandidates. It is to be noted that the scaling method performed to addthe global motion vector to the motion vector predictor candidates isthe same as the scaling method performed to add the temporal motionvector predictor to the motion vector predictor candidates except forusing, as an input, the global motion vector instead of a referencemotion vector for a co-located block.

FIG. 10A shows a method of deriving a motion vector predictor candidate(temporal motion vector predictor) using a forward reference motionvector in the temporal motion vector prediction mode when a co-locatedblock is a backward reference block and includes a forward referencemotion vector and a backward reference motion vector. More specifically,a motion vector predictor candidate (TemporalMV) is derived using theforward reference motion vector according to Expression 4 below.TemporalMV=mvL0×(B2−B0)/(B4−B0)  (Expression 4)

Here, (B2−B0) is time difference information indicating the differencebetween display time of a picture B2 and display time of a picture B0.Likewise, (B4−B0) is time difference information indicating thedifference between display time of a picture B4 and display time of apicture B0.

FIG. 10B shows a method for deriving a motion vector predictor candidate(temporal motion vector predictor) using the backward reference motionvector in the temporal motion vector predictor mode. More specifically,a motion vector predictor candidate is derived using the backwardreference motion vector according to Expression 5 below.TemporalMV=mvL1×(B2−B0)/(B4−B8)  (Expression 5)

FIG. 11A shows a method for deriving a motion vector predictor candidate(temporal motion vector predictor) using a backward reference motionvector in the temporal motion vector prediction mode when a co-locatedblock is a forward reference block and includes a forward referencemotion vector and a backward reference motion vector. More specifically,a motion vector predictor candidate is derived using the backwardreference motion vector according to Expression 6 below.TemporalMV=mvL1×(B6−B8)/(B4−B8)  (Expression 6)

FIG. 11B shows a method for deriving a motion vector predictor candidate(temporal motion vector predictor) using the forward reference motionvector in the temporal motion vector predictor mode. A motion vectorpredictor candidate is derived using the backward reference motionvector according to Expression 7 below.TemporalMV=mvL0×(B6−B8)/(B4−B0)  (Expression 7)

As described above, the present invention makes it possible to preventpropagation of a decoding error while suppressing decrease in codingefficiency by turning off, at constant intervals, the temporal motionvector predictor mode using a motion vector for a current coding unit ina reference picture and instead by adding, in header information, aglobal motion vector of the reference picture, and coding a motionvector for a coding target picture using the scaled global motionvector.

More specifically, when a co-located block use prohibition flag is on, aglobal motion vector read from the global vector storing unit 116 isadded in motion vector predictor candidates for the coding target blockand assigned to header information of a picture header or the like.Thus, even if a reference picture is lost in decoding, it is possible todecode the bitstream without being affected by the decoding error, andto thereby reduce error propagation.

In addition, when a co-located block use prohibition flag is off, it ispossible to select the reference motion vector optimum for the codingtarget block according to a co-located reference block direction flag,which makes it possible to increase the compression efficiency.Particularly when a co-located block is a forward reference block, it ispossible to reduce a prediction difference using a backward referencemotion vector. A backward reference motion vector is a motion vectorlocated in a direction from a picture including the co-located block tothe coding target block and thus has a high possibility of becomingcloser to the optimum motion vector. Therefore, the use of the backwardreference motion vector reduces the prediction difference.

On the other hand, a forward reference motion vector is a motion vectorlocated in the direction opposite to the direction from a pictureincluding the co-located block to the coding target block and thus has alow possibility of becoming closer to the optimum motion vector.Therefore, the use of the forward reference motion vector increases theprediction difference. In addition, also in the case where a co-locatedblock is a backward reference block, it is possible to yield a reducedprediction difference using a forward reference motion vector having ahigh possibility of becoming closer to the optimum motion vector.

In Embodiment 1, when the co-located block has two or more motionvectors, a switch is made between the motion vectors used for theco-located block to be used for calculating a temporal motion vectorpredictor for the coding target block, depending on whether theco-located block is a backward reference block or a forward referenceblock. However, this is a non-limiting example.

For example, it is also possible to calculate a temporal motion vectorpredictor using a motion vector which refers to the reference picturetemporally closest to the picture including the co-located block (themotion vector is a motion vector having a shortest temporal distance).Here, it is conceivable that the temporal distance is determineddepending on the number of pictures between the picture including theco-located block and the reference picture referred to by the co-locatedblock in display time order.

In Embodiment 1, when the co-located block has two or more motionvectors, a switch is made between the motion vectors used for theco-located block to be used for calculating a temporal motion vectorpredictor for the coding target block, depending on whether theco-located block is the backward reference block or the forwardreference block. However, this is a non-limiting example. For example,it is also possible to calculate a temporal motion vector predictorusing the smaller one of the two motion vectors for the co-locatedblock. Here, the magnitude of a motion vector means an absolute value orthe like of a motion vector.

In Embodiment 1, when the co-located block use prohibition flag is on,the global motion vector read from the global vector storing unit 116 isadded, as a replacement vector for a temporal motion vector predictor,to the motion vector predictor candidates. However, this is anon-limiting example. For example, it is also good to always determinethe value of the global motion vector to be 0 and add the global motionvector to motion vector predictor candidates (specifically, add themotion vector having a motion quantity of 0 as the replacement vector tothe motion vector predictor candidates). In this case, there is no needto assign header information or the like the global motion vector. Inaddition, when the co-located block use prohibition flag is on, it isalso good to always skip adding the temporal motion vector predictor inthe motion vector predictor candidates. Such skipping of inclusion ofthe temporal motion vector predictor in the motion vector predictorcandidates makes it possible to increase the coding efficiency.

In Embodiment 1, such a co-located block use prohibition flag may beassigned to only particular pictures instead of being assigned to eachof all the pictures. Examples of conceivable structures include astructure in which a co-located block use prohibition flag is assignedonly to each of pictures which are referred to by other pictures(P-pictures, B-pictures which are referred to by other pictures, andpictures belonging to the lowermost level layer in a reference structurecomposed of a plurality of layers) and no co-located block useprohibition flag is assigned to each of pictures which are not referredto by other pictures. As described above, it is possible to reducedecoding error propagation while increasing the coding efficiency byassigning the co-located block use prohibition flag to only each of theparticular pictures.

Although Embodiment 1 relates to the structure in which a co-locatedblock use prohibition flag is assigned for each picture, it is also goodto assign such a co-located block use prohibition flag for each slicecomposed of a plurality of blocks. Assignment of a co-located block useprohibition flag for each slice makes it possible to increase the globalvector prediction accuracy.

Although a co-located block use prohibition flag is assigned for each ofthe pictures in Embodiment 1, it is possible that no co-located blockuse prohibition flag is assigned and no temporal motion vector predictoris added to motion vector predictor candidates, based on a picture type,without assigning any co-located block use prohibition flag. Forexample, it is conceivable that a global vector predictor may be addedto motion vector predictor candidates without adding any temporal motionvector predictor to motion vector predictor candidates for each ofpictures which are referred to by other pictures (P-pictures, B-pictureswhich are referred to by other pictures, and pictures belonging to thelowermost level layer in a reference structure composed of a pluralityof layers). In this way, it is possible to omit such a co-located blockuse prohibition flag by determining whether or not to add the temporalmotion vector predictor to the motion vector predictor candidates, basedon the picture type. Therefore, it is possible to increase the codingefficiency.

Embodiment 2

FIG. 12 is a block diagram of a structure of a moving picture decodingapparatus 200 using a moving picture decoding method according toEmbodiment 2.

In Embodiment 2, a block included in a picture located forward of adecoding target picture in display time order is referred to as aforward reference block (the picture is a reference picture identifiedin a reference picture list L0). In addition, a block included in apicture located backward of the decoding target picture in display timeorder is referred to as a backward reference block (the picture is areference picture identified in a reference picture list L1).

As shown in FIG. 12, the moving picture decoding apparatus 200 includes:a variable length decoder 201, an inverse quantization unit 202, aninverse orthogonal transform unit 203, an adder 204, a block memory 205,a frame memory 206, an intra prediction unit 207, an inter predictionunit 208, a switch 209, an inter prediction control unit 210, a temporalmotion vector predictor calculating unit 211, and a colPic memory 212.

The variable length decoder 201: performs variable length decoding on aninput bitstream; outputs picture type information to the switch 209 andthe inter prediction control unit 210; outputs a motion vector predictorindex to the inter prediction control unit 210; outputs a co-locatedblock use prohibition flag and a co-located reference block directionflag, and a global motion vector to the temporal motion vector predictorcalculating unit 211; and outputs quantized coefficients to the inversequantization unit 202.

The inverse quantization unit 202 performs inverse quantization on thequantized coefficients obtained from the variable length decoder 201 toreconstruct transform coefficients, and outputs the reconstructedtransform coefficients to the inverse orthogonal transform unit 203. Theinverse orthogonal transform unit 203 transforms the reconstructedtransform coefficients obtained from the inverse quantization unit 202from a frequency domain to an image domain to reconstruct residualblocks, and outputs the reconstructed residual blocks to the adder 204.

The adder 204 adds the residual blocks obtained from the inverseorthogonal transform unit 203 and prediction blocks obtained from theswitch 209 to reconstruct decoded blocks. Next, the adder 204 outputs adecoded image stream including the reconstructed decoded blocks tooutside of the apparatus, and stores the decoded image stream in theblock memory 205 and the frame memory 206.

The block memory 205 stores, on a block-by-block basis, the decodedimage stream obtained from the adder 204. The frame memory 206 stores,on a frame-by-frame basis, the decoded image stream obtained from theadder 204.

The intra prediction unit 207 performs intra prediction using ablock-based decoded image stream stored in the block memory 205 togenerate a prediction block for each decoding target block, and outputsthe prediction block to the switch 209. The inter prediction unit 208performs inter prediction using a frame-based decoded image streamstored in the frame memory 206 to generate a prediction block for eachdecoding target block, and outputs the prediction block to the switch209. The switch 209 outputs, to the adder 204, the prediction blockgenerated by the intra prediction unit 207 and the prediction blockgenerated by the inter prediction unit 208.

When a co-located block use prohibition flag obtained from the variablelength decoder 201 is off, the temporal motion vector predictorcalculating unit 211 derives a motion vector predictor candidate(temporal motion vector predictor) in the temporal motion vectorpredictor mode, using colPic information such as a motion vector in aco-located block stored in the colPic memory 212. On the other hand,when the co-located block use prohibition flag is on, the temporalmotion vector predictor calculating unit 211 adds the global motionvector obtained from the variable length decoder 201 to motion vectorpredictor candidates.

In addition, the temporal motion vector predictor calculating unit 211assigns the motion vector predictor added to the candidates a motionvector predictor index. Next, the temporal motion vector predictorcalculating unit 211 outputs the motion vector predictor and the motionvector predictor index to the inter prediction control unit 210.

In addition, when the co-located block does not have any motion vector,the temporal motion vector predictor calculating unit 211 may stopmotion vector derivation in the temporal motion vector predictor mode oradd a motion vector having a motion quantity of 0 to motion vectorpredictor candidates.

The inter prediction control unit 210 determines a motion vectorpredictor corresponding to the motion vector predictor index obtainedfrom the variable length decoder 201 from among a plurality of motionvector predictor candidates. Next, the inter prediction control unit 210adds the determined motion vector predictor and information indicatingdifference between the motion vector and the motion vector predictor toderive a motion vector for use in inter prediction. In addition, theinter prediction control unit 210 stores, in the colPic memory 212,colPic information including a motion vector etc. for the decodingtarget block.

FIG. 13 is an outline of a processing flow of a moving picture decodingmethod according to Embodiment 2.

First, the variable length decoder 201 decodes the co-located block useprohibition flag for each picture (S81). Next, the variable lengthdecoder 201 determines whether or not the co-located block useprohibition flag is off (S82). When the co-located block use prohibitionflag is off (Yes in S82), the variable length decoder 201 decodes theco-located reference block direction flag for the picture (S83). Thevariable length decoder 201 then outputs the decoded co-located blockuse prohibition flag and co-located reference block direction flag tothe temporal motion vector predictor calculating unit 211.

Next, in the same manner as in FIG. 8, the temporal motion vectorpredictor calculating unit 211 reads colPic information including areference motion vector etc. in the co-located block from the colPicmemory 212 according to co-located block information, and adds atemporal motion vector predictor generated using the reference motionvector in the co-located block to motion vector predictor candidates(S84).

On the other hand, when the co-located block use prohibition flag is on,the temporal motion vector predictor calculating unit 211 obtains aglobal motion vector stored in header information such as a pictureheader from the variable length decoder 201, and adds the global motionvector to the motion vector predictor candidates (S87).

Next, the inter prediction control unit 210 determines a motion vectorpredictor corresponding to the decoded motion vector predictor indexfrom among the plurality of motion vector predictor candidates (S85). Inaddition, the inter prediction control unit 210 adds the determinedmotion vector predictor and the prediction difference information toderive a motion vector and outputs the motion vector to the interprediction unit 208. Next, the inter prediction unit 208 generates aprediction block for the decoding target block using the derived motionvector in inter prediction.

Next, the inter prediction control unit 210 stores, in the colPic memory212, colPic information including a motion vector etc. used in interprediction (S86). The colPic memory 212 stores a motion vector in areference picture, an index value of the reference picture, a predictiondirection etc. for calculation of a temporal motion vector predictor fora decoding target block.

The reference motion vector selecting method performed to calculate atemporal motion vector predictor when a reference block has two or morereference motion vectors is based on a co-located block use prohibitionflag. However, it is to be noted that the method is a non-limitingexample. For example, it is also good to calculate temporal distances ofreference motion vectors and use the one of the reference motion vectorswhich has the shortest temporal distance. Here, a temporal distance iscalculated based on the number of pictures between the reference pictureincluding a reference block and a picture which is referred to by thereference picture in display order. In addition, for example, it is alsogood to calculate the magnitudes of reference motion vectors anddetermine, to be a temporal motion vector predictor, the one of themotion vectors which has been derived using a small reference motionvector.

FIG. 14 shows examples of syntax of a bitstream in the moving picturedecoding method according to Embodiment 2. In FIG. 14,forbid_collocated_flag denotes a co-located block use prohibition flag,tmv_x denotes a horizontal component of a global motion vectorpredictor, tmv_y denotes a vertical component of the global motionvector predictor, and collocated_from_I0_flag denotes a co-locatedreference block direction flag.

As shown in FIG. 14, when the forbid_collocated_flag denoting theco-located block use prohibition flag is 1, the global motion vectorpredictors tmv_x and tmv_y are assigned to the bitstream and are addedto motion vector predictor candidates.

In addition, when the forbid_collocated_flag denoting the co-locatedblock use prohibition flag is 0, the collocated_from_I0_flag is assignedto the bitstream. In addition, a co-located block is determinedaccording to the co-located reference block direction flag, and atemporal motion vector predictor is calculated using the referencemotion vector in the co-located block. Here, a collocated_from_I0_flagof 1 shows that a co-located block is a forward co-located block and acollocated_from_I0_flag of 0 shows that a co-located block is a backwardco-located block. However, this is a non-limiting example.

In Embodiment 2, when the co-located block use prohibition flag is on,the global motion vector decoded from the header information or the likeis used. However, it is also good to always determine the value of aglobal motion vector predictor to be 0 in conformity with the codingmethod and add the global motion vector to motion vector predictorcandidates. In this case, the global motion vector is not assigned tothe header information or the like, and thus the decoding process isomitted. In addition, when the co-located block use prohibition flag ison, it is also good to always skip adding the temporal motion vectorpredictor to the motion vector predictor candidates.

As described above, in Embodiments 1 and 2, the temporal motion vectorpredictor mode using a motion vector for a current coding unit in areference picture is turned off at constant intervals, and instead aglobal motion vector of the reference picture is assigned to headerinformation. Using this header information, a motion vector for thecoding target picture is coded, which makes it possible to appropriatelydecode a bitstream with reduced decoding error propagation whilesuppressing decrease in coding efficiency.

More specifically, when a co-located block use prohibition flag is on,the global vector read from the global vector storing unit 116 is addedto the motion vector predictor candidates for the coding target blockand is assigned to the header information such as the picture headeretc. With this, even if a reference picture is lost in decoding, it ispossible to decode the bitstream without being affected by the decodingerror, and to thereby generate the bitstream with reduced error.

In addition, when a co-located block use prohibition flag is off, it ispossible to appropriately decode the biststream for which the referencemotion vector optimum for the coding target block has been selectedaccording to a co-located reference block direction flag.

Although the global vector read from the global vector storing unit 116is used when the co-located block use prohibition flag is on inEmbodiments 1 and 2, it is also good to always determine the value of aglobal motion vector to be 0 and add the global motion vector to motionvector predictor candidates. In addition, when the co-located block useprohibition flag is on, it is also good to always skip adding thetemporal motion vector predictor to the motion vector predictorcandidates. With this structure, it becomes possible to reduce theprocessing load for decoding.

In addition, although the co-located block use prohibition flags aredecoded from all the pictures in Embodiment 2, it is also good to decodesome co-located block use prohibition flags only from some particularpictures. For example, co-located block use prohibition flags aredecoded from only pictures which are referred to by other pictures(P-pictures, B-pictures which are referred to by other pictures, andpictures belonging to the lowermost level layer in a reference structurecomposed of a plurality of layers) and no co-located block useprohibition flag is decoded from pictures which are not referred to byother pictures. In this way, it is possible to reduce decoding errorpropagation while reducing the processing load for decoding, by decodingsome co-located block use prohibition flags from only particularpictures.

Although Embodiment 2 relates to the structure in which a co-locatedblock use prohibition flag is decoded for each picture, it is also goodto decode such a co-located block use prohibition flag for each slicecomposed of a plurality of blocks. Decoding of a co-located block useprohibition flag for each slice makes it possible to increase the globalvector prediction accuracy.

Although a co-located block use prohibition flag is decoded from each ofall the pictures in Embodiment 2, it is possible that no temporal motionvector predictor is added to motion vector predictor candidates, basedon a picture type. For example, it is conceivable that a global vectoris added to motion vector predictor candidates without adding anytemporal motion vector predictor to the motion vector predictorcandidates for each of pictures which are referred to by other pictures(P-pictures, B-pictures which are referred to by other pictures, andpictures belonging to the lowermost level layer in a reference structurecomposed of a plurality of layers). In this way, it is possible toincrease the coding efficiency while reducing the processing load fordecoding by determining whether to add a temporal motion vectorpredictor or a global motion vector to motion vector predictorcandidates, based on a picture type.

(Variation)

Next, a moving picture coding apparatus according to Variation ofEmbodiment 1 is described with reference to FIG. 15. FIG. 15 is a blockdiagram of a moving picture coding apparatus 300 according to Variationof Embodiment 1. The differences from Embodiment 1 are mainly describedbelow without repeating the same descriptions of the common points as inEmbodiment 1.

As shown in FIG. 15, the moving picture coding apparatus 300 includes afirst encoder 310 which codes a base view and outputs a resulting basebitstream and a second encoder 320 which codes a dependent view andoutput a resulting dependent bitstream. In the non-limiting exampleshown in FIG. 15, the base bitstream and the dependent bitstream areoutput independently of each other. However, it is also good to output asingle bitstream in which a base bitstream and a dependent bitstream arecombined.

The first and second encoders 310 and 320 have basically the samestructure as the equivalents in the moving picture coding apparatus 100shown in FIG. 1. However, the second encoder 320 can refer to a framememory 108 etc. of the first encoder 310, in addition to the referencedestinations in the structure in FIG. 1.

Next, a moving picture coding method according to Variation ofEmbodiment 1 is described with reference to FIGS. 16 and 17. FIG. 16 isa flowchart of operations in the moving picture coding method accordingto Variation of Embodiment 1. FIG. 17 is a diagram showing pictures in abase view and pictures in a dependent view.

As shown in FIG. 17, the base view includes a plurality of pictures I11,P12, P13, P14, I15, P16, and P17. In addition, among the pictures in thebase view, each of the starting pictures I11 and I15 in Groups OfPictures (GOPs) is an I-picture, and the other pictures P12, P13, P14,P16, and P17 are P-pictures. It is to be noted that the base view iscoded with reference to only pictures in the base view (in other words,using intra prediction coding or inter prediction coding), and isdecoded.

As shown in FIG. 17, the dependent view includes a plurality of picturesP21, P22, P23, P24, P25, P26, and P27. All of the pictures P21, P22,P23, P24, P25, P26, and P27 in the dependent view are P-pictures. It isto be noted that the dependent view is coded with reference to picturesin the dependent view and corresponding pictures in the base view (inother words, using inter-view prediction coding), and is decoded.

In addition, the base view and the dependent view are videos of asubject when viewed from different viewpoints. In other words, each ofpairs of mutually-corresponding pictures in the base view and thedependent view (having the same time stamp) has parallax in a horizontaldirection. The second encoder 320 can code each of the pictures in thedependent view, using a corresponding image in the base view as areference picture. Hereinafter, with reference to FIG. 16, a descriptionis given of operations performed by the temporal motion vector predictorcalculating unit 114 of the second encoder 320.

First, the temporal motion vector predictor calculating unit 114determines whether or not it is impossible to obtain a temporal motionvector predictor at the time of coding a coding target block (S91). Whenit is impossible to obtain any temporal motion vector predictor (Yes inS91), the temporal motion vector predictor calculating unit 114 includesa later-described parallax vector to motion vector predictor candidates(S92). On the other hand, when it is possible to obtain any temporalmotion vector predictor (No in S91), the temporal motion vectorpredictor calculating unit 114 includes a temporal motion vectorpredictor to the motion vector predictor candidates (S93).

Here, exemplary cases in which no temporal motion vector predictor canbe obtained includes a case where a coding target block is included inone of the starting pictures P21 and P25 in the GOPs. There are nopictures located forward of the starting pictures P21 and P25 in theGOPs in display time order and to be referred to by the startingpictures P21 and P25. In other words, when the coding order and thedisplay order match, the pictures P21 and P25 can refer to only thecorresponding pictures I11 and I15 in the base view, respectively.

However, the pictures I11 and I15 are I-pictures, and thus do not haveany motion vector information. In view of this, in such a case, thetemporal motion vector predictor calculating unit 114 includes, as areplacement vector for a temporal motion vector predictor, a parallaxvector stored in the global vector storing unit 116 in motion vectorpredictor candidates, and includes a parallax vector in headerinformation of the dependent bitstream.

Here, the parallax vector is a vector corresponding to parallax betweenthe base view and the dependent view. More specifically, the interprediction control unit 112 of the second encoder 320 outputs, to theglobal vector storing unit 116, a motion vector at the time ofperforming inter-view prediction coding on each of the blocks of thecoding target picture in the dependent view (in other words, the motionvector is a motion vector at the time of coding the correspondingpicture in the base view as a reference picture). The global vectorstoring unit 116 stores, as the parallax vector, an average value, amedian value, a mode value, or the like of motion vectors for eachpicture obtained from the inter prediction control unit 112.

In Step S92 of FIG. 16, the temporal motion vector predictor calculatingunit 114 may select, as the parallax vector for the picture P25 in thedependent view, (i) the parallax vector (whose reference picture is thepicture I11) calculated at the starting picture P21 in the GOPimmediately before the GOP including the picture P25 or (ii) theparallax vector (whose reference picture is the picture P14) calculatedat the picture P24 coded immediately before.

The above specific case where no temporal motion vector predictor can beobtained in Step S91 of FIG. 16 is a non-limiting example. As anotherexample case, the co-located block use prohibition flag for a codingtarget picture may be on. The co-located block use prohibition flag isthe same as in Embodiment 1, and thus the same description is notrepeated here.

As described above, this embodiment can be applied to the case of codingthe base view and the dependent view which constitute the multi-viewvideo. In other words, it is possible to prevent decoding errorpropagation while suppressing decrease in coding efficiency byselectively including a temporal motion vector predictor or a parallaxvector which is a replacement vector which replaces the temporal motionvector predictor, in motion vector predictor candidates.

Next, a moving picture decoding apparatus 400 according to Variation ofEmbodiment 2 is described with reference to FIG. 18. FIG. 18 is a blockdiagram of a moving picture decoding apparatus 400 according toVariation of Embodiment 2. The differences from Embodiment 2 are mainlydescribed below without repeating the same descriptions of the commonpoints as in Embodiment 2.

As shown in FIG. 18, the moving picture decoding apparatus 400 includesa first decoder 410 which decodes a base bitstream and outputs theresulting base view and a second decoder 420 which decodes a dependentbitstream and outputs the resulting dependent view. In the non-limitingexample shown in FIG. 18, the independent base bitstream and thedependent bitstream are input separately from each other. However, it isalso good to input a single bitstream in which a base bitstream and adependent bitstream are combined, and the bitstream is divided into thebase bitstream and the dependent bitstream inside the moving picturedecoding apparatus 400.

The first and second decoders 410 and 420 have basically the samestructures as the equivalents in the moving picture decoding apparatus200 shown in FIG. 12. However, the second decoder 420 can refer to aframe memory 206 etc. of the first decoder 410, in addition to thereference destinations in the structure in FIG. 12. In other words, themoving picture decoding apparatus 400 decodes the base bitstream and thedependent bitstream coded by the moving picture coding apparatus 300.

Next, the second decoder 420 of the moving picture decoding apparatus400 can selectively include, as a vector predictor for a decoding targetblock, a temporal motion vector predictor stored in the colPic memory212 or a parallax vector included in header information of the dependentbitstream.

Each of the structural elements in each of the above-describedembodiments may be configured in the form of an exclusive hardwareproduct, or may be realized by executing a software program suitable forthe structural element.

Each of the structural elements may be realized by means of a programexecuting unit, such as a CPU and a processor, reading and executing thesoftware program recorded on a recording medium such as a hard disk or asemiconductor memory. Here, the software programs for realizing themoving picture coding method and the moving picture decoding methodaccording to the embodiments are programs described below.

A program is for causing a computer to execute a moving picture codingmethod for performing inter prediction coding on a coding target blockincluded in a coding target picture, the moving picture coding methodincluding: coding the coding target block using a motion vector;generating a plurality of motion vector predictors; and coding themotion vector using one of the plurality of motion vector predictorsgenerated in the generating, wherein, in the generating, a replacementvector which replaces a temporal motion vector predictor is included inthe plurality of motion vector predictors when it is impossible toobtain the temporal motion vector predictor from a block which isincluded in a coded picture different from the coding target picture andcorresponds to the coding target block, and in the generating, a motionvector having a motion quantity of 0 is included, as the replacementvector, in the plurality of motion vector predictors when obtainment ofthe temporal motion vector predictor from the coded picture isprohibited.

In addition, a program is for causing a computer to execute a movingpicture decoding method for performing inter prediction decoding on adecoding target block included in a decoding target picture, the movingpicture decoding method including: decoding the decoding target blockusing a motion vector; generating a plurality of motion vectorpredictors; and decoding the motion vector using one of the plurality ofmotion vector predictors generated in the generating, wherein, in thegenerating, a replacement vector which replaces a temporal motion vectorpredictor is included in the plurality of motion vector predictors whenit is impossible to obtain the temporal motion vector predictor from ablock which is included in a decoded picture different from the decodingtarget picture and corresponds to the decoding target block, and in thegenerating, a motion vector having a motion quantity of 0 is included,as the replacement vector, in the plurality of motion vector predictorswhen obtainment of the temporal motion vector predictor from the decodedpicture is prohibited.

The herein disclosed subject matter is to be considered descriptive andillustrative only, and the appended Claims are of a scope intended tocover and encompass not only the particular embodiment(s) disclosed, butalso equivalent structures, methods, and/or uses.

Embodiment 3

The processing described in each of embodiments can be simplyimplemented in an independent computer system, by recording, in arecording medium, a program for implementing the configurations of themoving picture coding method (image coding method) and the movingpicture decoding method (image decoding method) described in each ofembodiments. The recording media may be any recording media as long asthe program can be recorded, such as a magnetic disk, an optical disk, amagnetic optical disk, an

IC card, and a semiconductor memory.

Hereinafter, the applications to the moving picture coding method (imagecoding method) and the moving picture decoding method (image decodingmethod) described in each of embodiments and systems using thereof willbe described. The system has a feature of having an image coding anddecoding apparatus that includes an image coding apparatus using theimage coding method and an image decoding apparatus using the imagedecoding method. Other configurations in the system can be changed asappropriate depending on the cases.

FIG. 20 illustrates an overall configuration of a content providingsystem ex100 for implementing content distribution services. The areafor providing communication services is divided into cells of desiredsize, and base stations ex106, ex107, ex108, ex109, and ex110 which arefixed wireless stations are placed in each of the cells.

The content providing system ex100 is connected to devices, such as acomputer ex111, a personal digital assistant (PDA) ex112, a cameraex113, a cellular phone ex114 and a game machine ex115, via the Internetex101, an Internet service provider ex102, a telephone network ex104, aswell as the base stations ex106 to ex110, respectively.

However, the configuration of the content providing system ex100 is notlimited to the configuration shown in FIG. 20, and a combination inwhich any of the elements are connected is acceptable. In addition, eachdevice may be directly connected to the telephone network ex104, ratherthan via the base stations ex106 to ex110 which are the fixed wirelessstations. Furthermore, the devices may be interconnected to each othervia a short distance wireless communication and others.

The camera ex113, such as a digital video camera, is capable ofcapturing video. A camera ex116, such as a digital camera, is capable ofcapturing both still images and video. Furthermore, the cellular phoneex114 may be the one that meets any of the standards such as GlobalSystem for Mobile Communications (GSM) (registered trademark), CodeDivision Multiple Access (CDMA), Wideband-Code Division Multiple Access(W-CDMA), Long Term Evolution (LTE), and High Speed Packet Access(HSPA). Alternatively, the cellular phone ex114 may be a PersonalHandyphone System (PHS).

In the content providing system ex100, a streaming server ex103 isconnected to the camera ex113 and others via the telephone network ex104and the base station ex109, which enables distribution of images of alive show and others. In such a distribution, a content (for example,video of a music live show) captured by the user using the camera ex113is coded as described above in each of embodiments (i.e., the camerafunctions as the image coding apparatus according to an aspect of thepresent disclosure), and the coded content is transmitted to thestreaming server ex103. On the other hand, the streaming server ex103carries out stream distribution of the transmitted content data to theclients upon their requests. The clients include the computer ex111, thePDA ex112, the camera ex113, the cellular phone ex114, and the gamemachine ex115 that are capable of decoding the above-mentioned codeddata. Each of the devices that have received the distributed datadecodes and reproduces the coded data (i.e., functions as the imagedecoding apparatus according to an aspect of the present disclosure).

The captured data may be coded by the camera ex113 or the streamingserver ex103 that transmits the data, or the coding processes may beshared between the camera ex113 and the streaming server ex103.Similarly, the distributed data may be decoded by the clients or thestreaming server ex103, or the decoding processes may be shared betweenthe clients and the streaming server ex103. Furthermore, the data of thestill images and video captured by not only the camera ex113 but alsothe camera ex116 may be transmitted to the streaming server ex103through the computer ex111. The coding processes may be performed by thecamera ex116, the computer ex111, or the streaming server ex103, orshared among them.

Furthermore, the coding and decoding processes may be performed by anLSI ex500 generally included in each of the computer ex111 and thedevices. The LSI ex500 may be configured of a single chip or a pluralityof chips. Software for coding and decoding video may be integrated intosome type of a recording medium (such as a CD-ROM, a flexible disk, anda hard disk) that is readable by the computer ex111 and others, and thecoding and decoding processes may be performed using the software.Furthermore, when the cellular phone ex114 is equipped with a camera,the video data obtained by the camera may be transmitted. The video datais data coded by the LSI ex500 included in the cellular phone ex114.

Furthermore, the streaming server ex103 may be composed of servers andcomputers, and may decentralize data and process the decentralized data,record, or distribute data.

As described above, the clients may receive and reproduce the coded datain the content providing system ex100. In other words, the clients canreceive and decode information transmitted by the user, and reproducethe decoded data in real time in the content providing system ex100, sothat the user who does not have any particular right and equipment canimplement personal broadcasting.

Aside from the example of the content providing system ex100, at leastone of the moving picture coding apparatus (image coding apparatus) andthe moving picture decoding apparatus (image decoding apparatus)described in each of embodiments may be implemented in a digitalbroadcasting system ex200 illustrated in FIG. 21. More specifically, abroadcast station ex201 communicates or transmits, via radio waves to abroadcast satellite ex202, multiplexed data obtained by multiplexingaudio data and others onto video data. The video data is data coded bythe moving picture coding method described in each of embodiments (i.e.,data coded by the image coding apparatus according to an aspect of thepresent disclosure). Upon receipt of the multiplexed data, the broadcastsatellite ex202 transmits radio waves for broadcasting. Then, a home-useantenna ex204 with a satellite broadcast reception function receives theradio waves. Next, a device such as a television (receiver) ex300 and aset top box (STB) ex217 decodes the received multiplexed data, andreproduces the decoded data (i.e., functions as the image decodingapparatus according to an aspect of the present disclosure).

Furthermore, a reader/recorder ex218 (i) reads and decodes themultiplexed data recorded on a recording medium ex215, such as a DVD anda BD, or (i) codes video signals in the recording medium ex215, and insome cases, writes data obtained by multiplexing an audio signal on thecoded data. The reader/recorder ex218 can include the moving picturedecoding apparatus or the moving picture coding apparatus as shown ineach of embodiments. In this case, the reproduced video signals aredisplayed on the monitor ex219, and can be reproduced by another deviceor system using the recording medium ex215 on which the multiplexed datais recorded. It is also possible to implement the moving picturedecoding apparatus in the set top box ex217 connected to the cable ex203for a cable television or to the antenna ex204 for satellite and/orterrestrial broadcasting, so as to display the video signals on themonitor ex219 of the television ex300. The moving picture decodingapparatus may be implemented not in the set top box but in thetelevision ex300.

FIG. 22 illustrates the television (receiver) ex300 that uses the movingpicture coding method and the moving picture decoding method describedin each of embodiments. The television ex300 includes: a tuner ex301that obtains or provides multiplexed data obtained by multiplexing audiodata onto video data, through the antenna ex204 or the cable ex203, etc.that receives a broadcast; a modulation/demodulation unit ex302 thatdemodulates the received multiplexed data or modulates data intomultiplexed data to be supplied outside; and amultiplexing/demultiplexing unit ex303 that demultiplexes the modulatedmultiplexed data into video data and audio data, or multiplexes videodata and audio data coded by a signal processing unit ex306 into data.

The television ex300 further includes: a signal processing unit ex306including an audio signal processing unit ex304 and a video signalprocessing unit ex305 that decode audio data and video data and codeaudio data and video data, respectively (which function as the imagecoding apparatus and the image decoding apparatus according to theaspects of the present disclosure); and an output unit ex309 including aspeaker ex307 that provides the decoded audio signal, and a display unitex308 that displays the decoded video signal, such as a display.Furthermore, the television ex300 includes an interface unit ex317including an operation input unit ex312 that receives an input of a useroperation. Furthermore, the television ex300 includes a control unitex310 that controls overall each constituent element of the televisionex300, and a power supply circuit unit ex311 that supplies power to eachof the elements. Other than the operation input unit ex312, theinterface unit ex317 may include: a bridge ex313 that is connected to anexternal device, such as the reader/recorder ex218; a slot unit ex314for enabling attachment of the recording medium ex216, such as an SDcard; a driver ex315 to be connected to an external recording medium,such as a hard disk; and a modem ex316 to be connected to a telephonenetwork. Here, the recording medium ex216 can electrically recordinformation using a non-volatile/volatile semiconductor memory elementfor storage. The constituent elements of the television ex300 areconnected to each other through a synchronous bus.

First, the configuration in which the television ex300 decodesmultiplexed data obtained from outside through the antenna ex204 andothers and reproduces the decoded data will be described. In thetelevision ex300, upon a user operation through a remote controllerex220 and others, the multiplexing/demultiplexing unit ex303demultiplexes the multiplexed data demodulated by themodulation/demodulation unit ex302, under control of the control unitex310 including a CPU. Furthermore, the audio signal processing unitex304 decodes the demultiplexed audio data, and the video signalprocessing unit ex305 decodes the demultiplexed video data, using thedecoding method described in each of embodiments, in the televisionex300. The output unit ex309 provides the decoded video signal and audiosignal outside, respectively. When the output unit ex309 provides thevideo signal and the audio signal, the signals may be temporarily storedin buffers ex318 and ex319, and others so that the signals arereproduced in synchronization with each other. Furthermore, thetelevision ex300 may read multiplexed data not through a broadcast andothers but from the recording media ex215 and ex216, such as a magneticdisk, an optical disk, and a SD card. Next, a configuration in which thetelevision ex300 codes an audio signal and a video signal, and transmitsthe data outside or writes the data on a recording medium will bedescribed. In the television ex300, upon a user operation through theremote controller ex220 and others, the audio signal processing unitex304 codes an audio signal, and the video signal processing unit ex305codes a video signal, under control of the control unit ex310 using thecoding method described in each of embodiments. Themultiplexing/demultiplexing unit ex303 multiplexes the coded videosignal and audio signal, and provides the resulting signal outside. Whenthe multiplexing/demultiplexing unit ex303 multiplexes the video signaland the audio signal, the signals may be temporarily stored in thebuffers ex320 and ex321, and others so that the signals are reproducedin synchronization with each other. Here, the buffers ex318, ex319,ex320, and ex321 may be plural as illustrated, or at least one buffermay be shared in the television ex300. Furthermore, data may be storedin a buffer so that the system overflow and underflow may be avoidedbetween the modulation/demodulation unit ex302 and themultiplexing/demultiplexing unit ex303, for example.

Furthermore, the television ex300 may include a configuration forreceiving an AV input from a microphone or a camera other than theconfiguration for obtaining audio and video data from a broadcast or arecording medium, and may code the obtained data. Although thetelevision ex300 can code, multiplex, and provide outside data in thedescription, it may be capable of only receiving, decoding, andproviding outside data but not the coding, multiplexing, and providingoutside data.

Furthermore, when the reader/recorder ex218 reads or writes multiplexeddata from or on a recording medium, one of the television ex300 and thereader/recorder ex218 may decode or code the multiplexed data, and thetelevision ex300 and the reader/recorder ex218 may share the decoding orcoding.

As an example, FIG. 23 illustrates a configuration of an informationreproducing/recording unit ex400 when data is read or written from or onan optical disk. The information reproducing/recording unit ex400includes constituent elements ex401, ex402, ex403, ex404, ex405, ex406,and ex407 to be described hereinafter. The optical head ex401 irradiatesa laser spot in a recording surface of the recording medium ex215 thatis an optical disk to write information, and detects reflected lightfrom the recording surface of the recording medium ex215 to read theinformation. The modulation recording unit ex402 electrically drives asemiconductor laser included in the optical head ex401, and modulatesthe laser light according to recorded data. The reproductiondemodulating unit ex403 amplifies a reproduction signal obtained byelectrically detecting the reflected light from the recording surfaceusing a photo detector included in the optical head ex401, anddemodulates the reproduction signal by separating a signal componentrecorded on the recording medium ex215 to reproduce the necessaryinformation. The buffer ex404 temporarily holds the information to berecorded on the recording medium ex215 and the information reproducedfrom the recording medium ex215. The disk motor ex405 rotates therecording medium ex215. The servo control unit ex406 moves the opticalhead ex401 to a predetermined information track while controlling therotation drive of the disk motor ex405 so as to follow the laser spot.The system control unit ex407 controls overall the informationreproducing/recording unit ex400. The reading and writing processes canbe implemented by the system control unit ex407 using variousinformation stored in the buffer ex404 and generating and adding newinformation as necessary, and by the modulation recording unit ex402,the reproduction demodulating unit ex403, and the servo control unitex406 that record and reproduce information through the optical headex401 while being operated in a coordinated manner. The system controlunit ex407 includes, for example, a microprocessor, and executesprocessing by causing a computer to execute a program for read andwrite.

Although the optical head ex401 irradiates a laser spot in thedescription, it may perform high-density recording using near fieldlight

FIG. 24 illustrates the recording medium ex215 that is the optical disk.On the recording surface of the recording medium ex215, guide groovesare spirally formed, and an information track ex230 records, in advance,address information indicating an absolute position on the diskaccording to change in a shape of the guide grooves. The addressinformation includes information for determining positions of recordingblocks ex231 that are a unit for recording data. Reproducing theinformation track ex230 and reading the address information in anapparatus that records and reproduces data can lead to determination ofthe positions of the recording blocks. Furthermore, the recording mediumex215 includes a data recording area ex233, an inner circumference areaex232, and an outer circumference area ex234. The data recording areaex233 is an area for use in recording the user data. The innercircumference area ex232 and the outer circumference area ex234 that areinside and outside of the data recording area ex233, respectively arefor specific use except for recording the user data. The informationreproducing/recording unit 400 reads and writes coded audio, coded videodata, or multiplexed data obtained by multiplexing the coded audio andvideo data, from and on the data recording area ex233 of the recordingmedium ex215.

Although an optical disk having a layer, such as a DVD and a BD isdescribed as an example in the description, the optical disk is notlimited to such, and may be an optical disk having a multilayerstructure and capable of being recorded on a part other than thesurface. Furthermore, the optical disk may have a structure formultidimensional recording/reproduction, such as recording ofinformation using light of colors with different wavelengths in the sameportion of the optical disk and for recording information havingdifferent layers from various angles.

Furthermore, a car ex210 having an antenna ex205 can receive data fromthe satellite ex202 and others, and reproduce video on a display devicesuch as a car navigation system ex211 set in the car ex210, in thedigital broadcasting system ex200. Here, a configuration of the carnavigation system ex211 will be a configuration, for example, includinga GPS receiving unit from the configuration illustrated in FIG. 22. Thesame will be true for the configuration of the computer ex111, thecellular phone ex114, and others.

FIG. 25A illustrates the cellular phone ex114 that uses the movingpicture coding method and the moving picture decoding method describedin embodiments. The cellular phone ex114 includes: an antenna ex350 fortransmitting and receiving radio waves through the base station ex110; acamera unit ex365 capable of capturing moving and still images; and adisplay unit ex358 such as a liquid crystal display for displaying thedata such as decoded video captured by the camera unit ex365 or receivedby the antenna ex350. The cellular phone ex114 further includes: a mainbody unit including an operation key unit ex366; an audio output unitex357 such as a speaker for output of audio; an audio input unit ex356such as a microphone for input of audio; a memory unit ex367 for storingcaptured video or still pictures, recorded audio, coded or decoded dataof the received video, the still pictures, e-mails, or others; and aslot unit ex364 that is an interface unit for a recording medium thatstores data in the same manner as the memory unit ex367.

Next, an example of a configuration of the cellular phone ex114 will bedescribed with reference to FIG. 25B. In the cellular phone ex114, amain control unit ex360 designed to control overall each unit of themain body including the display unit ex358 as well as the operation keyunit ex366 is connected mutually, via a synchronous bus ex370, to apower supply circuit unit ex361, an operation input control unit ex362,a video signal processing unit ex355, a camera interface unit ex363, aliquid crystal display (LCD) control unit ex359, amodulation/demodulation unit ex352, a multiplexing/demultiplexing unitex353, an audio signal processing unit ex354, the slot unit ex364, andthe memory unit ex367.

When a call-end key or a power key is turned ON by a user's operation,the power supply circuit unit ex361 supplies the respective units withpower from a battery pack so as to activate the cell phone ex114.

In the cellular phone ex114, the audio signal processing unit ex354converts the audio signals collected by the audio input unit ex356 invoice conversation mode into digital audio signals under the control ofthe main control unit ex360 including a CPU, ROM, and RAM. Then, themodulation/demodulation unit ex352 performs spread spectrum processingon the digital audio signals, and the transmitting and receiving unitex351 performs digital-to-analog conversion and frequency conversion onthe data, so as to transmit the resulting data via the antenna ex350.Also, in the cellular phone ex114, the transmitting and receiving unitex351 amplifies the data received by the antenna ex350 in voiceconversation mode and performs frequency conversion and theanalog-to-digital conversion on the data. Then, themodulation/demodulation unit ex352 performs inverse spread spectrumprocessing on the data, and the audio signal processing unit ex354converts it into analog audio signals, so as to output them via theaudio output unit ex357.

Furthermore, when an e-mail in data communication mode is transmitted,text data of the e-mail inputted by operating the operation key unitex366 and others of the main body is sent out to the main control unitex360 via the operation input control unit ex362. The main control unitex360 causes the modulation/demodulation unit ex352 to perform spreadspectrum processing on the text data, and the transmitting and receivingunit ex351 performs the digital-to-analog conversion and the frequencyconversion on the resulting data to transmit the data to the basestation ex110 via the antenna ex350. When an e-mail is received,processing that is approximately inverse to the processing fortransmitting an e-mail is performed on the received data, and theresulting data is provided to the display unit ex358.

When video, still images, or video and audio in data communication modeis or are transmitted, the video signal processing unit ex355 compressesand codes video signals supplied from the camera unit ex365 using themoving picture coding method shown in each of embodiments (i.e.,functions as the image coding apparatus according to the aspect of thepresent disclosure), and transmits the coded video data to themultiplexing/demultiplexing unit ex353. In contrast, during when thecamera unit ex365 captures video, still images, and others, the audiosignal processing unit ex354 codes audio signals collected by the audioinput unit ex356, and transmits the coded audio data to themultiplexing/demultiplexing unit ex353.

The multiplexing/demultiplexing unit ex353 multiplexes the coded videodata supplied from the video signal processing unit ex355 and the codedaudio data supplied from the audio signal processing unit ex354, using apredetermined method. Then, the modulation/demodulation unit(modulation/demodulation circuit unit) ex352 performs spread spectrumprocessing on the multiplexed data, and the transmitting and receivingunit ex351 performs digital-to-analog conversion and frequencyconversion on the data so as to transmit the resulting data via theantenna ex350.

When receiving data of a video file which is linked to a Web page andothers in data communication mode or when receiving an e-mail with videoand/or audio attached, in order to decode the multiplexed data receivedvia the antenna ex350, the multiplexing/demultiplexing unit ex353demultiplexes the multiplexed data into a video data bit stream and anaudio data bit stream, and supplies the video signal processing unitex355 with the coded video data and the audio signal processing unitex354 with the coded audio data, through the synchronous bus ex370. Thevideo signal processing unit ex355 decodes the video signal using amoving picture decoding method corresponding to the moving picturecoding method shown in each of embodiments (i.e., functions as the imagedecoding apparatus according to the aspect of the present disclosure),and then the display unit ex358 displays, for instance, the video andstill images included in the video file linked to the Web page via theLCD control unit ex359. Furthermore, the audio signal processing unitex354 decodes the audio signal, and the audio output unit ex357 providesthe audio.

Furthermore, similarly to the television ex300, a terminal such as thecellular phone ex114 probably have 3 types of implementationconfigurations including not only (i) a transmitting and receivingterminal including both a coding apparatus and a decoding apparatus, butalso (ii) a transmitting terminal including only a coding apparatus and(iii) a receiving terminal including only a decoding apparatus. Althoughthe digital broadcasting system ex200 receives and transmits themultiplexed data obtained by multiplexing audio data onto video data inthe description, the multiplexed data may be data obtained bymultiplexing not audio data but character data related to video ontovideo data, and may be not multiplexed data but video data itself.

As such, the moving picture coding method and the moving picturedecoding method in each of embodiments can be used in any of the devicesand systems described. Thus, the advantages described in each ofembodiments can be obtained.

Furthermore, various modifications and revisions can be made in any ofthe embodiments in the present disclosure.

Embodiment 4

Video data can be generated by switching, as necessary, between (i) themoving picture coding method or the moving picture coding apparatusshown in each of embodiments and (ii) a moving picture coding method ora moving picture coding apparatus in conformity with a differentstandard, such as MPEG-2, MPEG-4 AVC, and VC-1.

Here, when a plurality of video data that conforms to the differentstandards is generated and is then decoded, the decoding methods need tobe selected to conform to the different standards. However, since towhich standard each of the plurality of the video data to be decodedconforms cannot be detected, there is a problem that an appropriatedecoding method cannot be selected.

In order to solve the problem, multiplexed data obtained by multiplexingaudio data and others onto video data has a structure includingidentification information indicating to which standard the video dataconforms. The specific structure of the multiplexed data including thevideo data generated in the moving picture coding method and by themoving picture coding apparatus shown in each of embodiments will behereinafter described. The multiplexed data is a digital stream in theMPEG-2 Transport Stream format.

FIG. 26 illustrates a structure of the multiplexed data. As illustratedin FIG. 26, the multiplexed data can be obtained by multiplexing atleast one of a video stream, an audio stream, a presentation graphicsstream (PG), and an interactive graphics stream. The video streamrepresents primary video and secondary video of a movie, the audiostream (IG) represents a primary audio part and a secondary audio partto be mixed with the primary audio part, and the presentation graphicsstream represents subtitles of the movie. Here, the primary video isnormal video to be displayed on a screen, and the secondary video isvideo to be displayed on a smaller window in the primary video.Furthermore, the interactive graphics stream represents an interactivescreen to be generated by arranging the GUI components on a screen. Thevideo stream is coded in the moving picture coding method or by themoving picture coding apparatus shown in each of embodiments, or in amoving picture coding method or by a moving picture coding apparatus inconformity with a conventional standard, such as MPEG-2, MPEG-4 AVC, andVC-1. The audio stream is coded in accordance with a standard, such asDolby-AC-3, Dolby Digital Plus, MLP, DTS, DTS-HD, and linear PCM.

Each stream included in the multiplexed data is identified by PID. Forexample, 0x1011 is allocated to the video stream to be used for video ofa movie, 0x1100 to 0x111F are allocated to the audio streams, 0x1200 to0x121F are allocated to the presentation graphics streams, 0x1400 to0x141F are allocated to the interactive graphics streams, 0x1B00 to0x1B1F are allocated to the video streams to be used for secondary videoof the movie, and 0x1A00 to 0x1A1F are allocated to the audio streams tobe used for the secondary audio to be mixed with the primary audio.

FIG. 27 schematically illustrates how data is multiplexed. First, avideo stream ex235 composed of video frames and an audio stream ex238composed of audio frames are transformed into a stream of PES packetsex236 and a stream of PES packets ex239, and further into TS packetsex237 and TS packets ex240, respectively. Similarly, data of apresentation graphics stream ex241 and data of an interactive graphicsstream ex244 are transformed into a stream of PES packets ex242 and astream of PES packets ex245, and further into TS packets ex243 and TSpackets ex246, respectively. These TS packets are multiplexed into astream to obtain multiplexed data ex247.

FIG. 28 illustrates how a video stream is stored in a stream of PESpackets in more detail. The first bar in FIG. 28 shows a video framestream in a video stream. The second bar shows the stream of PESpackets. As indicated by arrows denoted as yy1, yy2, yy3, and yy4 inFIG. 28, the video stream is divided into pictures as I pictures, Bpictures, and P pictures each of which is a video presentation unit, andthe pictures are stored in a payload of each of the PES packets. Each ofthe PES packets has a PES header, and the PES header stores aPresentation Time-Stamp (PTS) indicating a display time of the picture,and a Decoding Time-Stamp (DTS) indicating a decoding time of thepicture.

FIG. 29 illustrates a format of TS packets to be finally written on themultiplexed data. Each of the TS packets is a 188-byte fixed lengthpacket including a 4-byte TS header having information, such as a PIDfor identifying a stream and a 184-byte TS payload for storing data. ThePES packets are divided, and stored in the TS payloads, respectively.When a BD ROM is used, each of the TS packets is given a 4-byteTP_Extra_Header, thus resulting in 192-byte source packets. The sourcepackets are written on the multiplexed data. The TP_Extra_Header storesinformation such as an Arrival_Time_Stamp (ATS). The ATS shows atransfer start time at which each of the TS packets is to be transferredto a PID filter. The source packets are arranged in the multiplexed dataas shown at the bottom of FIG. 29. The numbers incrementing from thehead of the multiplexed data are called source packet numbers (SPNs).

Each of the TS packets included in the multiplexed data includes notonly streams of audio, video, subtitles and others, but also a ProgramAssociation Table (PAT), a Program Map Table (PMT), and a Program ClockReference (PCR). The PAT shows what a PID in a PMT used in themultiplexed data indicates, and a PID of the PAT itself is registered aszero. The PMT stores PIDs of the streams of video, audio, subtitles andothers included in the multiplexed data, and attribute information ofthe streams corresponding to the PIDs. The PMT also has variousdescriptors relating to the multiplexed data. The descriptors haveinformation such as copy control information showing whether copying ofthe multiplexed data is permitted or not. The PCR stores STC timeinformation corresponding to an ATS showing when the PCR packet istransferred to a decoder, in order to achieve synchronization between anArrival Time Clock (ATC) that is a time axis of ATSs, and an System TimeClock (STC) that is a time axis of PTSs and DTSs.

FIG. 30 illustrates the data structure of the PMT in detail. A PMTheader is disposed at the top of the PMT. The PMT header describes thelength of data included in the PMT and others. A plurality ofdescriptors relating to the multiplexed data is disposed after the PMTheader. Information such as the copy control information is described inthe descriptors. After the descriptors, a plurality of pieces of streaminformation relating to the streams included in the multiplexed data isdisposed. Each piece of stream information includes stream descriptorseach describing information, such as a stream type for identifying acompression codec of a stream, a stream PID, and stream attributeinformation (such as a frame rate or an aspect ratio). The streamdescriptors are equal in number to the number of streams in themultiplexed data.

When the multiplexed data is recorded on a recording medium and others,it is recorded together with multiplexed data information files.

Each of the multiplexed data information files is management informationof the multiplexed data as shown in FIG. 31. The multiplexed datainformation files are in one to one correspondence with the multiplexeddata, and each of the files includes multiplexed data information,stream attribute information, and an entry map.

As illustrated in FIG. 31, the multiplexed data information includes asystem rate, a reproduction start time, and a reproduction end time. Thesystem rate indicates the maximum transfer rate at which a system targetdecoder to be described later transfers the multiplexed data to a PIDfilter. The intervals of the ATSs included in the multiplexed data areset to not higher than a system rate. The reproduction start timeindicates a PTS in a video frame at the head of the multiplexed data. Aninterval of one frame is added to a PTS in a video frame at the end ofthe multiplexed data, and the PTS is set to the reproduction end time.

As shown in FIG. 32, a piece of attribute information is registered inthe stream attribute information, for each PID of each stream includedin the multiplexed data. Each piece of attribute information hasdifferent information depending on whether the corresponding stream is avideo stream, an audio stream, a presentation graphics stream, or aninteractive graphics stream. Each piece of video stream attributeinformation carries information including what kind of compression codecis used for compressing the video stream, and the resolution, aspectratio and frame rate of the pieces of picture data that is included inthe video stream. Each piece of audio stream attribute informationcarries information including what kind of compression codec is used forcompressing the audio stream, how many channels are included in theaudio stream, which language the audio stream supports, and how high thesampling frequency is. The video stream attribute information and theaudio stream attribute information are used for initialization of adecoder before the player plays back the information.

In the present embodiment, the multiplexed data to be used is of astream type included in the PMT. Furthermore, when the multiplexed datais recorded on a recording medium, the video stream attributeinformation included in the multiplexed data information is used. Morespecifically, the moving picture coding method or the moving picturecoding apparatus described in each of embodiments includes a step or aunit for allocating unique information indicating video data generatedby the moving picture coding method or the moving picture codingapparatus in each of embodiments, to the stream type included in the PMTor the video stream attribute information. With the configuration, thevideo data generated by the moving picture coding method or the movingpicture coding apparatus described in each of embodiments can bedistinguished from video data that conforms to another standard.

Furthermore, FIG. 33 illustrates steps of the moving picture decodingmethod according to the present embodiment. In Step exS100, the streamtype included in the PMT or the video stream attribute informationincluded in the multiplexed data information is obtained from themultiplexed data. Next, in Step exS101, it is determined whether or notthe stream type or the video stream attribute information indicates thatthe multiplexed data is generated by the moving picture coding method orthe moving picture coding apparatus in each of embodiments. When it isdetermined that the stream type or the video stream attributeinformation indicates that the multiplexed data is generated by themoving picture coding method or the moving picture coding apparatus ineach of embodiments, in Step exS102, decoding is performed by the movingpicture decoding method in each of embodiments. Furthermore, when thestream type or the video stream attribute information indicatesconformance to the conventional standards, such as MPEG-2, MPEG-4 AVC,and VC-1, in Step exS103, decoding is performed by a moving picturedecoding method in conformity with the conventional standards.

As such, allocating a new unique value to the stream type or the videostream attribute information enables determination whether or not themoving picture decoding method or the moving picture decoding apparatusthat is described in each of embodiments can perform decoding. Even whenmultiplexed data that conforms to a different standard is input, anappropriate decoding method or apparatus can be selected. Thus, itbecomes possible to decode information without any error. Furthermore,the moving picture coding method or apparatus, or the moving picturedecoding method or apparatus in the present embodiment can be used inthe devices and systems described above.

Embodiment 5

Each of the moving picture coding method, the moving picture codingapparatus, the moving picture decoding method, and the moving picturedecoding apparatus in each of embodiments is typically achieved in theform of an integrated circuit or a Large Scale Integrated (LSI) circuit.As an example of the LSI, FIG. 34 illustrates a configuration of the LSIex500 that is made into one chip. The LSI ex500 includes elements ex501,ex502, ex503, ex504, ex505, ex506, ex507, ex508, and ex509 to bedescribed below, and the elements are connected to each other through abus ex510. The power supply circuit unit ex505 is activated by supplyingeach of the elements with power when the power supply circuit unit ex505is turned on.

For example, when coding is performed, the LSI ex500 receives an AVsignal from a microphone ex117, a camera ex113, and others through an AVIO ex509 under control of a control unit ex501 including a CPU ex502, amemory controller ex503, a stream controller ex504, and a drivingfrequency control unit ex512. The received AV signal is temporarilystored in an external memory ex511, such as an SDRAM. Under control ofthe control unit ex501, the stored data is segmented into data portionsaccording to the processing amount and speed to be transmitted to asignal processing unit ex507. Then, the signal processing unit ex507codes an audio signal and/or a video signal. Here, the coding of thevideo signal is the coding described in each of embodiments.Furthermore, the signal processing unit ex507 sometimes multiplexes thecoded audio data and the coded video data, and a stream IO ex506provides the multiplexed data outside. The provided multiplexed data istransmitted to the base station ex107, or written on the recordingmedium ex215. When data sets are multiplexed, the data should betemporarily stored in the buffer ex508 so that the data sets aresynchronized with each other.

Although the memory ex511 is an element outside the LSI ex500, it may beincluded in the LSI ex500. The buffer ex508 is not limited to onebuffer, but may be composed of buffers. Furthermore, the LSI ex500 maybe made into one chip or a plurality of chips.

Furthermore, although the control unit ex501 includes the CPU ex502, thememory controller ex503, the stream controller ex504, the drivingfrequency control unit ex512, the configuration of the control unitex501 is not limited to such. For example, the signal processing unitex507 may further include a CPU. Inclusion of another CPU in the signalprocessing unit ex507 can improve the processing speed. Furthermore, asanother example, the CPU ex502 may serve as or be a part of the signalprocessing unit ex507, and, for example, may include an audio signalprocessing unit. In such a case, the control unit ex501 includes thesignal processing unit ex507 or the CPU ex502 including a part of thesignal processing unit ex507.

The name used here is LSI, but it may also be called IC, system LSI,super LSI, or ultra LSI depending on the degree of integration.

Moreover, ways to achieve integration are not limited to the LSI, and aspecial circuit or a general purpose processor and so forth can alsoachieve the integration. Field Programmable Gate Array (FPGA) that canbe programmed after manufacturing LSIs or a reconfigurable processorthat allows re-configuration of the connection or configuration of anLSI can be used for the same purpose. Such a programmable logic devicecan typically execute the moving picture coding method and/or the movingpicture decoding method according to any of the above embodiments, by,loading or reading from a memory or the like one or more programs thatare included in software or firmware.

In the future, with advancement in semiconductor technology, a brand-newtechnology may replace LSI. The functional blocks can be integratedusing such a technology. The possibility is that the present disclosureis applied to biotechnology.

Embodiment 6

When video data generated in the moving picture coding method or by themoving picture coding apparatus described in each of embodiments isdecoded, compared to when video data that conforms to a conventionalstandard, such as MPEG-2, MPEG-4 AVC, and VC-1 is decoded, theprocessing amount probably increases. Thus, the LSI ex500 needs to beset to a driving frequency higher than that of the CPU ex502 to be usedwhen video data in conformity with the conventional standard is decoded.However, when the driving frequency is set higher, there is a problemthat the power consumption increases.

In order to solve the problem, the moving picture decoding apparatus,such as the television ex300 and the LSI ex500 is configured todetermine to which standard the video data conforms, and switch betweenthe driving frequencies according to the determined standard. FIG. 35illustrates a configuration ex800 in the present embodiment. A drivingfrequency switching unit ex803 sets a driving frequency to a higherdriving frequency when video data is generated by the moving picturecoding method or the moving picture coding apparatus described in eachof embodiments. Then, the driving frequency switching unit ex803instructs a decoding processing unit ex801 that executes the movingpicture decoding method described in each of embodiments to decode thevideo data. When the video data conforms to the conventional standard,the driving frequency switching unit ex803 sets a driving frequency to alower driving frequency than that of the video data generated by themoving picture coding method or the moving picture coding apparatusdescribed in each of embodiments. Then, the driving frequency switchingunit ex803 instructs the decoding processing unit ex802 that conforms tothe conventional standard to decode the video data.

More specifically, the driving frequency switching unit ex803 includesthe CPU ex502 and the driving frequency control unit ex512 in FIG. 34.Here, each of the decoding processing unit ex801 that executes themoving picture decoding method described in each of embodiments and thedecoding processing unit ex802 that conforms to the conventionalstandard corresponds to the signal processing unit ex507 in FIG. 34. TheCPU ex502 determines to which standard the video data conforms. Then,the driving frequency control unit ex512 determines a driving frequencybased on a signal from the CPU ex502. Furthermore, the signal processingunit ex507 decodes the video data based on the signal from the CPUex502. For example, the identification information described inEmbodiment 4 is probably used for identifying the video data. Theidentification information is not limited to the one described inEmbodiment 4 but may be any information as long as the informationindicates to which standard the video data conforms. For example, whenwhich standard video data conforms to can be determined based on anexternal signal for determining that the video data is used for atelevision or a disk, etc., the determination may be made based on suchan external signal. Furthermore, the CPU ex502 selects a drivingfrequency based on, for example, a look-up table in which the standardsof the video data are associated with the driving frequencies as shownin FIG. 37. The driving frequency can be selected by storing the look-uptable in the buffer ex508 and in an internal memory of an LSI, and withreference to the look-up table by the CPU ex502.

FIG. 36 illustrates steps for executing a method in the presentembodiment. First, in Step exS200, the signal processing unit ex507obtains identification information from the multiplexed data. Next, inStep exS201, the CPU ex502 determines whether or not the video data isgenerated by the coding method and the coding apparatus described ineach of embodiments, based on the identification information. When thevideo data is generated by the moving picture coding method and themoving picture coding apparatus described in each of embodiments, inStep exS202, the CPU ex502 transmits a signal for setting the drivingfrequency to a higher driving frequency to the driving frequency controlunit ex512. Then, the driving frequency control unit ex512 sets thedriving frequency to the higher driving frequency. On the other hand,when the identification information indicates that the video dataconforms to the conventional standard, such as MPEG-2, MPEG-4 AVC, andVC-1, in Step exS203, the CPU ex502 transmits a signal for setting thedriving frequency to a lower driving frequency to the driving frequencycontrol unit ex512. Then, the driving frequency control unit ex512 setsthe driving frequency to the lower driving frequency than that in thecase where the video data is generated by the moving picture codingmethod and the moving picture coding apparatus described in each ofembodiment.

Furthermore, along with the switching of the driving frequencies, thepower conservation effect can be improved by changing the voltage to beapplied to the LSI ex500 or an apparatus including the LSI ex500. Forexample, when the driving frequency is set lower, the voltage to beapplied to the LSI ex500 or the apparatus including the LSI ex500 isprobably set to a voltage lower than that in the case where the drivingfrequency is set higher.

Furthermore, when the processing amount for decoding is larger, thedriving frequency may be set higher, and when the processing amount fordecoding is smaller, the driving frequency may be set lower as themethod for setting the driving frequency. Thus, the setting method isnot limited to the ones described above. For example, when theprocessing amount for decoding video data in conformity with MPEG-4 AVCis larger than the processing amount for decoding video data generatedby the moving picture coding method and the moving picture codingapparatus described in each of embodiments, the driving frequency isprobably set in reverse order to the setting described above.

Furthermore, the method for setting the driving frequency is not limitedto the method for setting the driving frequency lower. For example, whenthe identification information indicates that the video data isgenerated by the moving picture coding method and the moving picturecoding apparatus described in each of embodiments, the voltage to beapplied to the LSI ex500 or the apparatus including the LSI ex500 isprobably set higher. When the identification information indicates thatthe video data conforms to the conventional standard, such as MPEG-2,MPEG-4 AVC, and VC-1, the voltage to be applied to the LSI ex500 or theapparatus including the LSI ex500 is probably set lower. As anotherexample, when the identification information indicates that the videodata is generated by the moving picture coding method and the movingpicture coding apparatus described in each of embodiments, the drivingof the CPU ex502 does not probably have to be suspended. When theidentification information indicates that the video data conforms to theconventional standard, such as MPEG-2, MPEG-4 AVC, and VC-1, the drivingof the CPU ex502 is probably suspended at a given time because the CPUex502 has extra processing capacity. Even when the identificationinformation indicates that the video data is generated by the movingpicture coding method and the moving picture coding apparatus describedin each of embodiments, in the case where the CPU ex502 has extraprocessing capacity, the driving of the CPU ex502 is probably suspendedat a given time. In such a case, the suspending time is probably setshorter than that in the case where when the identification informationindicates that the video data conforms to the conventional standard,such as MPEG-2, MPEG-4 AVC, and VC-1.

Accordingly, the power conservation effect can be improved by switchingbetween the driving frequencies in accordance with the standard to whichthe video data conforms. Furthermore, when the LSI ex500 or theapparatus including the LSI ex500 is driven using a battery, the batterylife can be extended with the power conservation effect.

Embodiment 7

There are cases where a plurality of video data that conforms todifferent standards, is provided to the devices and systems, such as atelevision and a cellular phone. In order to enable decoding theplurality of video data that conforms to the different standards, thesignal processing unit ex507 of the LSI ex500 needs to conform to thedifferent standards. However, the problems of increase in the scale ofthe circuit of the LSI ex500 and increase in the cost arise with theindividual use of the signal processing units ex507 that conform to therespective standards.

In order to solve the problem, what is conceived is a configuration inwhich the decoding processing unit for implementing the moving picturedecoding method described in each of embodiments and the decodingprocessing unit that conforms to the conventional standard, such asMPEG-2, MPEG-4 AVC, and VC-1 are partly shared. Ex900 in FIG. 38A showsan example of the configuration. For example, the moving picturedecoding method described in each of embodiments and the moving picturedecoding method that conforms to MPEG-4 AVC have, partly in common, thedetails of processing, such as entropy coding, inverse quantization,deblocking filtering, and motion compensated prediction. The details ofprocessing to be shared probably include use of a decoding processingunit ex902 that conforms to MPEG-4 AVC. In contrast, a dedicateddecoding processing unit ex901 is probably used for other processingwhich is unique to an aspect of the present disclosure and does notconform to MPEG-4 AVC. Since the aspect of the present disclosure ischaracterized by decoding of motion vectors in particular, for example,the dedicated decoding processing unit ex901 is used for decoding ofmotion vectors. Otherwise, the decoding processing unit is probablyshared for one of the entropy decoding, deblocking filtering, motioncompensation, and inverse quantization, or all of the processing. Thedecoding processing unit for implementing the moving picture decodingmethod described in each of embodiments may be shared for the processingto be shared, and a dedicated decoding processing unit may be used forprocessing unique to that of MPEG-4 AVC.

Furthermore, ex1000 in FIG. 38B shows another example in that processingis partly shared. This example uses a configuration including adedicated decoding processing unit ex1001 that supports the processingunique to an aspect of the present disclosure, a dedicated decodingprocessing unit ex1002 that supports the processing unique to anotherconventional standard, and a decoding processing unit ex1003 thatsupports processing to be shared between the moving picture decodingmethod according to the aspect of the present disclosure and theconventional moving picture decoding method. Here, the dedicateddecoding processing units ex1001 and ex1002 are not necessarilyspecialized for the processing according to the aspect of the presentdisclosure and the processing of the conventional standard,respectively, and may be the ones capable of implementing generalprocessing. Furthermore, the configuration of the present embodiment canbe implemented by the LSI ex500.

As such, reducing the scale of the circuit of an LSI and reducing thecost are possible by sharing the decoding processing unit for theprocessing to be shared between the moving picture decoding methodaccording to the aspect of the present disclosure and the moving picturedecoding method in conformity with the conventional standard.

INDUSTRIAL APPLICABILITY

The moving picture coding method and the moving picture decoding methodaccording to the one or more exemplary embodiments of the presentdisclosure are used advantageously for moving picture coding apparatusesand moving picture decoding apparatuses.

The invention claimed is:
 1. A moving picture decoding apparatuscomprising: a processor; and a memory having a computer program storedthereon, the computer program causing the processor to executeoperations including: obtaining identification information indicatingwhether a bit stream is coded by a first standard or coded by a secondstandard from the bit stream, the first standard being different fromthe second standard; when the identification information indicating thebit stream is coded by the first standard, decoding the bit streamaccording to the first standard; and when the identification informationindicating the bit stream is coded by the second standard, obtaining atemporal motion vector enable flag which indicates whether or not usinga temporal motion vector predictor is enabled, the temporal motionvector predictor being from a block which is included in a decodedpicture different from a decoding target picture and which correspondsto a decoding target block included in the decoding target picture,generating a plurality of motion vector predictors, decoding a motionvector using one of the plurality of motion vector predictors generatedin the generating, and decoding the decoding target block using themotion vector decoded in the decoding of the motion vector, wherein, inthe generating, a motion vector having a motion quantity of 0 isgenerated when the temporal motion vector enable flag indicates thatusing the temporal motion vector predictor is not enabled, the motionvector being included, as a replacement vector which replaces thetemporal motion vector predictor, in the plurality of motion vectorpredictors.